Method for locating a vehicle within a map in which at least one line is drawn

Abstract

The invention relates to a method for locating a vehicle (1) within a map (14) in which at least one line (6) is drawn. The method comprises: determining (S2) a first line information item (11), which at least partially describes at least one at least partially line-shaped object (7) in an environment of the vehicle (1) by applying a line determination criterion (12) to an environment information item (10) that describes at least part of the environment of the vehicle (1); determining (S3) a second line information item (13), which at least partially describes the at least one line (6) drawn in the map (14), by analysing the map (14); and locating (S4) the vehicle (1) within the map (14) by carrying out an assignment procedure (15), in which the respective line-shaped object (7) described by the determined first line information item (11) is assigned precisely to the line (6) described by the determined second line information item (13), which is drawn into the map (14) for the line-shaped object (7).

Description

[0001]2023PF00341 1 Method for locating a vehicle within a map in which at least one line is drawn The invention relates to a method for locating a vehicle within a map in which at least one line is drawn. Furthermore, the invention relates to a vehicle, a control device for a vehicle and a computer program product for carrying out such a method. A vehicle can, for example, have a driver assistance system which can assume longitudinal and / or lateral guidance of the vehicle at least partially automatically, in particular fully automatically, while the vehicle is parking in or out of a parking space. Such a driver assistance system can specify that the vehicle must first be localized, for example by locating it on a map. The map can, for example, describe a parking garage or other parking environment.For localization, the vehicle, for example, uses a sensor device to detect its current surroundings and associates objects detected in the surroundings with objects marked on the map. However, if the sensor device is a single camera, for example, distortions may occur regarding the imaging of objects in the vehicle's surroundings. It can therefore be difficult to reliably associate certain objects with the objects marked on the map. EP 2601481 B1 shows a method for assisting a parking process in a parking garage using a parking system in a vehicle. The parking garage comprises a first evaluation and control unit and provides information about the parking garage to a control unit of the vehicle. This information includes geometric data of the parking garage, which includes position information and / or dimensional information of walls, restricted areas and / or pillars.The object of the invention is to provide a solution by means of which a vehicle can be reliably located within a map. This object is achieved by the subject matter of the independent patent claims. A first aspect of the invention relates to a method for locating a vehicle within a map. At least one line is drawn on the map. The 2023PF00341 2 map is stored, for example, in the vehicle, for example in a storage device. The storage device is, for example, comprised by a control device of the vehicle, or the control device has access to the storage device. Alternatively or additionally, the map can be transmitted to the vehicle, for example by an external computing device.The external computing device is, for example, a parking space management device that provides a map that describes different areas of a parking environment, in particular different parking levels or parking zones of the parking environment. The parking environment is, for example, a parking garage or a parking area comprising several individual parking spaces. The at least one line can then, for example, be a parking space marking of a parking space within the parking environment. In the map, therefore, for example, parking space markings are drawn as lines. The method comprises determining first line information. The first line information at least partially describes an at least partially linear object in an environment of the vehicle. The first line information is determined by applying a line determination criterion to an environment information item. The environment information describes at least part of the environment of the vehicle.The environmental information is acquired, for example, by means of a sensor device of the vehicle, in particular by means of a camera device of the vehicle. The acquired environmental information is provided to the control device of the vehicle, i.e., transmitted to it internally in the vehicle, for example, so that the control device can then determine the first line information. In the case of the camera device as a sensor device, the environmental information can alternatively be referred to as camera information. This describes a static and / or moving image or picture of the environment. The camera device of the vehicle can be or comprise a front camera, a side camera, a rear camera and / or a surround view camera. The first line information describes, for example, the parking space marking of a parking space in the parking environment when the vehicle is arranged in a parking environment.The parking space marking is, for example, a stripe on the ground in the parking area and is therefore a ground marking. The at least partially linear object can therefore be at least part of the parking space marking. The line determination criterion is an algorithm and / or comprises at least one rule, the implementation or application of which uses the environmental information and thus, for example, raw data from the sensor device 2023PF00341 3 to determine where in the environmental information and thus, for example, raw data from the sensor device 2023PF00341 3 the at least partially linear object is located. Object recognition methods can be used for this purpose, for example an algorithm that can recognize and identify various objects related to a parking space or vehicle. Furthermore, it can be provided that it is known in advance that the determined first piece of line information describes a parking space marking.The method comprises determining a second piece of line information by evaluating the map. The second piece of line information at least partially describes the at least one line drawn on the map. Thus, not only are actually recognizable lines in the surroundings, i.e. the at least one linear object, determined, but the at least one line is also taken from the map. For this purpose, the line determination criterion and / or a deviating map line determination criterion can be applied to the map in order to determine the second piece of line information. The map line determination criterion can have an algorithm and / or at least one rule by means of which lines drawn on the map can be determined.It can be assumed that the first line information may be distorted, bent, and / or locally shifted compared to the second line information, for example, due to a curvature of the ground on which the vehicle is standing, imaging properties of the sensor device, and / or other possible influences on the environmental information. It is also assumed below that, for example, positioning the vehicle on the map, for example using a position-determining device based on a global navigation satellite system (GNSS), is not sufficiently accurate, which is why visual localization of the vehicle should be performed based on the line information.The method comprises locating the vehicle within the map by performing an assignment method in which the respective linear object described by the determined first line information is assigned precisely to the line described by the determined second line information, which is plotted on the map for the linear object. When performing the assignment method, for example, a comparison can be made of at least one section and / or point of the lines from the map and at least one section and / or point of the detected 2023PF00341 4 linear object in the surroundings in order to be able to infer a relative position and / or a relative orientation of the vehicle to the map. The assignment method can alternatively be referred to as an assignment criterion or assignment algorithm.The assignment method is, for example, part of a computer program that assigns the respective detected linear object according to the first line information to a corresponding line drawn in the map according to the second line information. This enables the localization of the vehicle within the map. It is assumed, for example, that a position and orientation of the sensor device relative to a coordinate system of the vehicle are known. The described method is preferably carried out by means of the vehicle's control device and thus within the vehicle. The method according to the invention is, in particular, a computer-implemented method.It can be provided that the vehicle, for example, transmits the environmental information to the parking space management device and / or another external computing device, such as a backend, a server, a cloud server and / or another vehicle, so that the vehicle to be located is localized outside of the vehicle and localization information describing a result of the localization is transmitted to the vehicle. Vehicle-to-vehicle or vehicle-to-infrastructure communication can be used for this purpose. The method shows how the vehicle can be localized within the map without, for example, having to correct optical imaging errors and / or the influences of uneven ground. This is possible because the linear object and the line drawn on the map are taken into account.The described method is particularly suitable for individual cameras, i.e. mono cameras, as the sensor device that provides the environmental information. By applying the method, a vehicle as a whole can always be reliably located within a map. One embodiment of the invention provides that the first line information describes an angle at which an object starting point of the at least one linear object is oriented to the sensor device of the vehicle that detects the environmental information. The first line information therefore does not describe, for example, the position of the linear object in the surroundings of the vehicle or relative to the vehicle, but describes the angle at which light was detected that describes the linear object 2023PF00341 5. The angle therefore describes the angle from which a beam or light ray, starting from the object starting point, hits the sensor device.The entire linear object is not always taken into account, but only its starting point, which is referred to here as the object start point. The object start point is the point assigned to the linear object that is closest to the sensor device. The reason for taking the object start point into account is that, especially with a mono camera as the sensor device, reliable position information for individual pixels cannot be determined. The further away a pixel is from the sensor device, the less accurate the information about its position becomes. Therefore, only the starting point of the linear object is taken into account here, and not parts of the linear object that are further away. Furthermore, the angle is taken into account because, in comparison to the position relative to the sensor device, it can be determined precisely. This is precisely what is used here, with the first piece of line information describing the described angle.The method is therefore particularly suitable for the mono camera as a sensor device. A further exemplary embodiment provides that the second piece of line information describes a line starting point of the at least one line that is drawn on the map. Thus, not the entire line drawn on the map is taken into account, but rather the starting point of this line. The line starting point is the point on the line at which the line begins or ends. It can be taken into account that, for example, a vehicle located in a park environment initially drives on an area intended for driving within the park environment. This area is, for example, a street adjacent to parking spaces and / or an entrance area of ​​the park environment. The starting point of the line is then, for example, the end of the line that faces the area intended for driving, i.e. is located closest to it.Alternatively or additionally, the vehicle can first be positioned in the parking area using the position determination device and at least roughly located on the map. It can then be determined which end of the respective line is closest to the vehicle and should therefore be assumed to be the line starting point. It can therefore be assumed that it is known which end of the line and thus which line starting point is closest to the vehicle and should therefore be assigned to the respective linear object according to the first piece of line information. Therefore, neither the entire linear objects nor the entire lines are considered to carry out the localization. This simplifies the process so that it can be carried out quickly and accurately. 2023PF00341 6 If, for example, there are several lines on the map, several line starting points are described by the second piece of line information.Analogously, different angles for the various linear objects detected from the environmental information are described by the first piece of line information if multiple linear objects are detectable in the environment. If, for example, five linear objects were detected, the first piece of line information comprises five angles. These remain unchanged relative to one another. Analogously, the second piece of line information can describe multiple line starting points, in particular their relative positions to one another. When carrying out the assignment method, each of the multiple angles is assigned to exactly one of the multiple line starting points, wherein relative positions and / or orientations of the individual angles and / or line starting points to one another can be taken into account. Overall, this makes the method particularly reliable.Another embodiment provides that when carrying out the assignment method, a brute force method is first applied, by means of which a range of relative positions and / or relative orientations of the respective linear object to the respective line is determined by trial and error. A problem-solving method from the field of computer science is therefore used, in which, by trial and error of various possible cases, it is determined how the first piece of line information is positioned and / or oriented relative to the second piece of line information. A result obtained in this way may contain an error, wherein the error lies, in particular, within a predetermined error range. By applying the brute force method, an approximation to an exact solution for the relative position and / or the relative orientation is thus achieved. The exact solution has, for example, a minimal error compared to other solutions.For example, the at least one angle described by the first piece of line information and the at least one line starting point described by the second piece of line information are taken into account, and these two pieces of information are related to one another until, for example, the at least one angle matches the at least one line starting point in terms of its position and / or orientation, or vice versa. A range of possible relative positions and / or relative orientations is first determined, for which it is assumed, for example, that the first and second pieces of line information match one another. For example, a first scan can be carried out over all possible relative positions and / or relative orientations in order to at least approximate the actual relative position and / or relative orientation.The first and second pieces of line information can, for example, each be plotted on a grid or a map and, when applying the brute-force methods, moved relative to one another in predetermined steps, i.e., shifted and / or rotated, in order to determine in which relative positions and / or relative orientations they best match one another compared to the other tested relative positions and / or relative orientations. The match can be assessed based on whether, for example, a respective straight line lying at a respective angle to the vehicle, in particular to the sensor device, intersects one of the line start points. Thus, an assignment takes place between the first and second pieces of line information in the x- and y-directions as well as in an angular direction. This makes it clear how the vehicle can be located, at least roughly.An additional embodiment provides that a distance between two tried-out relative positions and / or relative orientations is initially set to a first distance value during the application of the brute force method. During the application of the brute force method, the distance is reduced, in particular in steps. Alternatively or additionally, the distance can be reduced continuously. The reduction occurs until and / or until a second distance value is reached. The first and second distance values ​​are different. The second distance value is smaller than the first distance value. Thus, for example, the brute force method can initially be carried out with a grid of 1 meter, i.e. with a distance of 1 meter between two tried-out relative positions. Alternatively or additionally, the distance between two tried-out orientations can be, for example, 3 degrees.After testing with this distance, the range of relative positions and / or relative orientations is determined, for example, by finding the best results, i.e., the best possible correlation of the two line information items compared to other tested relative positions and / or relative orientations. The grid is then reduced in size and applied again to the determined area. In other words, the determined area is zoomed in. For the next test, the distance between two tested relative positions and / or relative orientations is reduced compared to the previous one, for example, to 0.5 meters and / or 1 degree. A sub-area with the best results is determined for the area searched in this way.This sub-area is then the newly determined area, in which relative positions 2023PF00341 8 and / or relative orientations are then tried again using a reduced-size grid. This is repeated until the second distance value is reached, which is, for example, 20 centimeters and / or 0.5 degrees. Thus, initially a relatively coarse localization of the vehicle is carried out, followed by an increasingly fine localization, until ultimately a relatively small area of ​​possible relative positions and / or relative orientations remains. The brute-force method thus enables quick and easy pre-localization of the vehicle within the map. Furthermore, one embodiment provides for a numerical optimization method to be applied after the brute-force method has been applied when carrying out the assignment process.This is applied in particular as soon as the distance between two tested relative positions and / or relative orientations corresponds to the second distance value. The numerical optimization method can, for example, be based on the Gauss-Newton method. Alternative and / or additional methods or algorithms from the field of numerical optimization can be applied. By applying the numerical optimization method, an optimal parameter of a relatively complex system is generally found, i.e., an objective function is minimized or maximized. According to the invention, for example, the relative position and / or relative orientation is determined at which the environment described by the environmental information optimally matches the map. This is the case when the probability of a coincidence or assignment of the linear objects in the environment and the lines drawn on the map is maximized.For example, the brute-force method is first used to roughly perform the assignment and then switched to the numerical optimization method, which is more precise than the brute-force method. This is then only applied locally to the area determined during the last application of the brute-force method. This results in minimal time expenditure, as the numerical optimization method is only required in a spatially limited area. This also saves resources, as the numerical optimization method is typically more resource-intensive than the brute-force method. Furthermore, it can be avoided that, for example, a local maximum is found when applying the numerical optimization method rather than the actually sought global maximum, at which the overall probability of the linear objects matching the respective lines on the map is maximum.2023PF00341 9 As soon as the overall best relative position and / or relative orientation has been found, for example the global maximum, it is possible to calculate where and how the vehicle is oriented in the map, taking into account the position of the sensor device in the vehicle, so that its localization is achieved. Furthermore, one embodiment provides that when carrying out the assignment method, a probability value is determined for each relative position and / or relative orientation and / or is taken from a list of predefined probability values. The respective probability value describes a probability with which the respective linear object is plotted as a specific line of, for example, several lines in the map.The respective probability value is based, for example, on a standard deviation and thus on an assumed error that can be assigned to each relative position and / or relative orientation. For example, for each possible position and / or orientation, for example of the first piece of line information relative to the second piece of line information, a value or a function is determined that indicates the probability that, for this position and / or orientation, the first piece of line information is assigned to the second piece of line information or vice versa. The probability value can be calculated or, for example, calculated in advance for possible first pieces of line information and stored in the list. The list is stored, for example, in the memory device of the control device. Alternatively or additionally, the list can be transmitted from the external computing device to the vehicle, in particular at the request of the vehicle.This makes it clear how it can be decided which relative position and / or relative orientation is interpreted as a meaningful and therefore good result of the assignment method and which is not. For this purpose, for example, the probability value can be maximized. Furthermore, one embodiment provides that the probability value is determined by determining and evaluating a distance between the line starting point of the at least one line drawn on a map and a ray that strikes the sensor device at the respective angle described by the first line information. For example, it is checked how far away the line starting point is from the ray that strikes the sensor device or emanates from it at the angle according to the first line information.This is a quick and straightforward calculation for estimating the probability value, as it may, for example, depend on the stated distance, which can be calculated with little effort. This avoids a complex calculation, which is typically based on solving an arctangent. Another exemplary embodiment provides that the probability value is determined by determining and evaluating an angle between the line starting point of the at least one line drawn on the map and a ray that strikes the sensor device at the respective angle described by the first line information. Such a calculation is based on geometric considerations and therefore on calculating an arctangent. This calculation is therefore significantly more complex than the previously described distance-based determination of the probability value.However, in this determination of the probability value, an explicit measurement function is given that is solved and can be calculated directly from the available parameters, i.e., depending on the relative position and / or the relative orientation and the line information. In contrast, the previously described determination of the probability value is based on solving an implicit measurement function. Therefore, various methods are useful for reliably determining the probability value. Furthermore, one embodiment provides that, when determining the probability value, a calibration error of the sensor device, an error in the line determination criterion, and / or an error in a sensor device model assumed for the sensor device are taken into account.In the case of a camera as the sensor device, the sensor device model is, for example, the camera model or lens model used for this camera to evaluate and provide the environmental information. The uncertainty with which the environmental information is provided is therefore taken into account, be it due to the calibration error and / or the error in the sensor device model. In addition or alternatively, it is taken into account that the line determination criterion that determines the first line information may be error-prone, for example due to the quality of the model used for this purpose. The model used is based, for example, on machine learning, in particular on an artificial neural network, so that a quality can be known for the model used. The uncertainty of the detection of the linear object in the vehicle's surroundings is thus at least taken into account.This means, for example, that the further the object starting point is from the sensor device, the more error the angle as the first piece of line information can have. Different probability values ​​are therefore possible for object starting points 2023PF00341 11 that are relatively far away from the sensor device compared to object starting points that are closer to the sensor device. This in turn influences the implementation of the assignment method and ultimately the determination of the relative position and / or relative orientation at which the maximum probability value is determined. It is therefore particularly useful to at least partially take the aforementioned errors into account. Alternative or further errors or inaccuracies can be taken into account. A further exemplary embodiment provides that the at least one at least partially linear object is or represents a parking space marking of a parking space in the surrounding area.It can therefore be firmly provided that the method for locating the vehicle in a parking environment in which the linear objects are always, for example, floor markings of individual parking spaces. Alternatively or additionally, the parking space marking can be formed at least partially by a curb, a wall and / or another type of straight object. The method is thus particularly suitable for a parking assistant or a parking aid that localizes the vehicle in the parking environment. The parking assistant can, for example, be designed to follow a previously trained parking trajectory at least partially automatically, in particular fully automatically. Furthermore, one exemplary embodiment provides that the at least one line drawn on the map describes or indicates a parking space marking of a parking space in a parking environment.In this case, the map contains precisely the lines identified as linear objects, enabling a direct association between the drawn lines and linear objects. This allows the focus, particularly in the parking environment, to be precisely on the structures in the map that are relevant for accurately locating the vehicle. The reason for this is that parking space markings are generally fixed and thus static, and also provide useful information about possible parking destinations for the vehicle. Alternative or additional linear objects or drawn lines are possible if the method is to be used to locate the vehicle in other environments. A further aspect of the invention relates to a vehicle that is designed to carry out the method described above.The vehicle is preferably a motor vehicle, in particular a passenger car, a truck, a bus, a motorcycle, and / or a moped. The vehicle has, for example, the control device 2023PF00341 12 for carrying out the method described above. In addition, the vehicle can comprise the sensor device, such as the camera or camera device. An additional aspect of the invention relates to a control device for a vehicle. The control device is designed to carry out the steps of the method described above. The control device is, for example, an on-board computer of the vehicle. In order to carry out the method, the environmental information detected by the sensor device is available to the control device, i.e., this information has already been transmitted to the control device. In addition, the control device has access to the map.The control device carries out the method described above, in particular an exemplary embodiment or a combination of exemplary embodiments of the method. The control device has a processor device. The processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). Furthermore, the processor device can have program code, which can alternatively be referred to as a computer program product. The program code can be stored in a data memory of the processor device. An additional aspect of the invention relates to a computer program product. The computer program product is a computer program.The computer program product comprises instructions which, when the program is executed by a computer, such as by the control devices, cause the computer to carry out the steps of the method according to the invention. The exemplary embodiments described in connection with the method according to the invention, each individually and in combination with one another, apply accordingly, where applicable, to the vehicle according to the invention, the control device according to the invention and the computer program product according to the invention. The invention comprises combinations of the described exemplary embodiments. Therein: Fig. 1 shows a schematic representation of a vehicle in a parking environment; 2023PF00341 13 Fig. 2 shows a schematic representation of a signal flow graph of a method for locating a vehicle within a map; Fig. 3 shows a schematic representation of a vehicle relative to a line; Fig.Fig. 4 shows a further schematic representation of a vehicle in relation to a line; Fig. 5 shows a schematic representation of a mounting position of a sensor device in a vehicle; Fig. 6 shows a schematic representation of a radiation probability field around a vehicle; Fig. 7 shows a schematic representation of an evaluation of the method for four different vehicle positions; Fig. 8 shows a schematic representation of an estimation of a maximum probability of a vehicle position in relation to line starting points on a map; and Fig. 9 shows a schematic representation of the determination of a probability value. In the figures, functionally identical elements are provided with the same reference numerals. Fig. 1 shows a vehicle 1 which is located in a parking environment 4. The vehicle 1 here has two sensor devices 2, each of which is designed as a camera mounted on the side of the vehicle 1.The two sensor devices 2 can be arranged in side mirrors of the vehicle 1. The vehicle 1 also has a control device 3. By means of this, data recorded by the respective sensor device 2 can be evaluated. In the parking environment 4 there are several parking spaces 5. The parking space 5 can alternatively be referred to as a parking area in the parking environment 4. Here, for example, several parking spaces 5 arranged next to one another are sketched to the right and left of the vehicle 1 in the direction of travel. A map 14 (see reference numeral 14 in Fig. 2) of the parking environment 4 is available here, on which the parking space markings, shown here as solid lines 6, are drawn. Each parking space 5 is bounded laterally by such a line 6. In addition, individual linear objects 7 are sketched here, which show the parking space markings from a perspective of the respective sensor device 2.Due, for example, to uneven ground and / or imaging effects and / or imaging errors of the respective sensor device 2, the linear objects 7 are at least partially oriented and / or positioned differently than the actual parking space markings according to the line 6 in the map 14. Furthermore, rays 8 are sketched that describe the exact direction from which the light strikes the sensor device 2, which describes an object starting point of the respective linear object 7. It is therefore known at which angle 21 (see reference numeral 21 in Fig. 3) the sensor device 2 detects the respective linear object 7. The invention is directed to assigning the respective linear object 7 precisely to one of the lines 6 from the map 14, so that the vehicle 1 from which the linear objects 7 were detected can be localized within the map 14. Fig.2 shows method steps of a method for locating a vehicle 1 within the map 14, wherein at least one of the lines 6 is drawn in the map 14. In a method step S1, it can be provided that, by means of the respective sensor device 2, environmental information 10 is first acquired, which describes at least part of the environment of the vehicle 1. The environment here is the parking environment 4. Alternative environments are possible. The sensor device 2 here is the side camera. Alternatively or additionally, it can be a front camera and / or rear camera and / or a surround view camera of the vehicle 1. The sensor device 2 provides the environmental information 10 to a control device 3, for example by transmitting it to the latter, so that the method steps S2 to S4 described below can be carried out by means of the control device 3 of the vehicle 1.2023PF00341 15 In a method step S2, a first piece of line information 11 is determined, which at least partially describes at least one at least partially linear object 7 in the surroundings of the vehicle 1. The first piece of line information 11 can, for example, describe the angle 21 at which an object starting point of the respective linear object 7 is oriented to the sensor device 2 of the vehicle 1. For example, the first piece of line information 11 describes the plurality of rays 8 shown in Fig. 1. Each of these rays 8 is then defined by the angle 21. The first piece of line information 11 is determined by applying a line determination criterion 12 to the surroundings information 10. In a method step S3, a second piece of line information 13 is determined, which at least partially describes the at least one line 6 shown in the map 14. For this purpose, the map 14 is evaluated.The second line information 13 here describes a line starting point 20 (see reference numeral 20 in Fig. 3) of the at least one line 6 that is plotted on the map 14. In a method step S4, the vehicle 1 is localized within the map 14 by performing an assignment method 15, in which the respective linear object 7 described by the determined first line information 11 is assigned precisely to the line 6 described by the determined second line information 13, which is plotted for the linear object 7 on the map 14. Thus, for example, the plurality of linear objects 7 are each assigned to the plurality of lines 6 in the map 14, so that ultimately, for each of the linear objects 7 plotted above in Fig. 1, the line 6 in the map 14 that actually describes this linear object 7 is found, or vice versa.The assignment method 15 can first apply a brute force method 16 and then a numerical optimization method 17. It can be provided that a range of relative positions and / or relative orientations of the respective linear object 7 to the respective line 6 is first determined by trial and error using the brute force method 16, wherein a match is found between the linear object 7 and line 6 in the range with a high probability. A distance between two tried-out relative positions and / or relative orientations is initially set to a first distance value and is repeatedly reduced during the application of the brute force method 16, in particular stepwise, until a second distance value is reached.In other words, for example, the parking area 4 is first scanned with relative position changes of 1 2023PF00341 16 meters and relative orientation changes of several degrees, for example, 3 or 4 degrees. Once this has been done and a maximum has been found, the brute force method 16 is applied again in an area around this maximum with a lower resolution, for example, by reducing the distance between two tested relative positions to half a meter and / or the distance between two tested relative orientations to 1 degree. This can be repeated until the second distance value is reached. This is, for example, a relative position distance of 20 centimeters.It can therefore be provided that a range of 20 centimeters by 20 centimeters is identified in which the relative position is expected in which the linear objects 7 can be optimally assigned to the respective lines 6 in the map 14. Analogously, a relative orientation range can be identified. Once the range has been determined, a switch can be made from the brute force method 16 to a numerical optimization method 17, whereby this only occurs when the distance between two tested relative positions and / or relative orientations corresponds to the second distance value, for example, 20 centimeters for the relative position.When performing the assignment method 15, a probability value can be determined for each tested relative position and / or each tested relative orientation and / or taken from a list of predefined probability values, wherein the respective probability value describes a probability with which the linear object 7 is plotted as the line 6 in the map 14. The calculation required for localization is discussed in detail below and derived: To avoid a systematic error in localization, we propose using the angular information (angle 21) of the detected lines 6. This information can be taken directly from an image space and is thus the true measurement of the camera as the sensor device 2.This is done by aligning the rays 8, which pass through a camera position, with the object start point and thus the start points of the linear objects 7, as shown in Fig.1. 2023PF00341 17 How well a certain ray angle α (hereinafter used for the angle 21) matches the line start point 20 of a certain line 6 can generally be expressed as a so-called innovation function, which relates the vehicle position X of the vehicle 1 to. as an argument, so that the innovation function is: The simplest and most intuitive innovation function is the angular difference between the angle α and the angle 21 of the object's starting point relative to the camera, as sketched in Fig. 3. However, this has the disadvantage of requiring the plotting of an arctangent, which, as explained later, can be runtime-intensive. Therefore, an alternative form of the innovation function will be used below, namely the distance of ray 8 to the line starting point 20 of line 6, as shown in Fig. 4. The innovation function can then be written as follows: Since the camera position in the world view (x c W , y c W ) T a function of the vehicle position X and the installation position of the camera (x c , y c ) T , it becomes clear that the innovation function ν is a function of X. This is illustrated by Fig.5 and the following formulas: 2023PF00341 18 It should be noted that this innovation function is implicit, since it is not possible to express Z as a function of X. To obtain a model for the probability of a measurement, the innovation function is linearized around a measurement expectation Ẑ: Assuming that the measurement Z is normally distributed and R is a covariance matrix, P(ν=0|X) is also normally distributed, as follows: where N{0;0m;P} is the value assumed by a normal distribution with mean m and covariance P at point 0. Note that this result is not trivial and assumes independence between Z and X. However, this model does not consider the possibility that the measurement Z does not originate from the map point (xs, ys) Tbut is simply a noise signal. This can be included in the probability determination by summing the normal distribution with a constant noise probability ^ (this is a direct application of the total probability theorem): Since a probability can be scaled by any multiplication factor and the value of ^ is arbitrary, the value of P(Clutter) is irrelevant and can be ignored, yielding: 2023PF00341 19 Assuming: equation (2.8) can be rewritten as: This shows that the modeling of noise signals can be achieved by modifying the squared error δ(X) using a specific cost function C ≥ 0. Equation (2.9) is called the Blake-Zisserman cost function; other possibilities are known in the literature and will be presented later. Here, the cost function C is assumed to be the Blake-Zisserman cost function. Furthermore, the fact that the correspondence of a line 6 in the map 14 to a ray 8 detected by the camera is unknown is taken into account. Let m be the number of rays 8 detected by the camera and n the number of lines 6. A r j Let the event be that the ray 8 (r) starts from a certain line starting point 20 (j), where for 1 ≤ r ≤ m and 1 ≤ j ≤ n: be the measurement vector for the ray r, the line starting point j, the covariance matrix R r and the squared error of Z j r : 2023PF00341 20 Again, using the fact that the probability is only interesting up to a certain multiplication factor, the fact that all P(Ar j ) can be chosen so that they are equally likely as P(Ar j ) = 1 / m and the fact that ^ is arbitrary, we get: Now the multivariate Blake-Zisserman cost function is introduced as follows: The probability for a line starting point 20 is given by: Due to the independence between the line starting points 20, their total probability is: Using equation (2.17), we can now find the maximum likelihood estimate and the corresponding covariance. This begins by constructing a probability field 22 containing equation 2.14. All cells are initialized with the clutter probability ^. Then, for each ray r, a rectangle is formed around the 3σ distance perpendicular to line 6. This 3σ distance increases with the distance to the camera, just as the Jacobian matrix Jj r increases with distance. Furthermore, a minimum and maximum detection distance or range of the camera is assumed to be limited. Now, for the cells within the rectangle, the following is added to the original clutter probability: 2023PF00341 21 This can be done for all cells of the grid, but further than the 3σ distance, the contribution becomes almost 0 and would therefore consume more propagation time without any additional benefit. This process is repeated for all rays 8 and is shown in Fig. 6. This shows a schematic representation of an example ray probability field around vehicle 1. This illustrates the advantage of using the implicit measurement model instead of constructing the innovation function dependent on angle 21, as it avoids the calculation of an arctangent, which would otherwise be required to calculate the probability of all cells. The cos and sin values ​​can be calculated in advance, and the pre-calculation of the probability field 22 therefore does not require any trigonometric operation.Using the probability field 22, for a given vehicle position X, the position of all line starting points 20 in the coordinate system of the probability field 22 can be calculated and the probability of these line starting points 20 can be obtained. The same search method can therefore be used with multiple resolutions. This results in the following steps: 1. Determining the probability field 22 in fine resolution, as previously described. 2. Discretizing the state space X in θ, x, and y and calculating the value of equation (2.17) for each discretized value of X (for each voxel). 3. Using three For loops, the discretization in θ is used in the outermost For loop, since rotation is the most expensive operation and should be reused for different x, y, and map features. The two inner For loops increment x and y with a fixed step size.To determine the position of the line starting points 20 within the probability field 22, simple division is sufficient. 4. For a given X, the value taken by equation (2.17) is the product of all values ​​obtained from the pre-calculated probability field 22 for each line starting point 20. 2023PF00341 22 This procedure is illustrated in Fig. 7, in which four different relative positions A, B, CD for four lines 6 and rays 8 are sketched. The position at which equation (2.17) has its maximum is called the maximum likelihood estimate (ML for maximum likelihood) and is sketched in Fig. 8. This shows a schematic representation of an estimate of the maximum probability of a vehicle position with respect to line starting points 20 from map 14. The covariance of this estimate can be determined by fitting a Gaussian function over the sample values ​​of equation (2.17).One can start by normalizing the probability P(ν =0|X) over the entire voxel space, where pV is the normalized probability of a particular voxel V. It can then be either the exact expected value. as the mean value for the Gaussian function or are estimated using the maximum likelihood Let X Mthe mean of the Gaussian function. The covariance of the Gaussian function is obtained by summing over all voxels: . The advantage of this method for evaluating covariance is that it provides very realistic estimates, as it considers the shape of the probability over a large area and not just locally. In practice, however, ML estimation is preferred because it is close to the expected value and is known to be more robust in terms of localization accuracy than the expected value. However, the accuracy of the search for the ML estimate is limited by the granularity of the search. Therefore, we use a search with several different resolutions. The probability field 22 is created at high resolution, and a second probability field 22 with lower resolution is also created (decreasing distances between relative positions and / or relative orientations).At low resolution, details can disappear, so the value of a low-resolution cell is set to the maximum value of the high-resolution cells that contain it. This ensures that the probabilities calculated by the low-resolution map 14 are always at least as large as those in the high-resolution probability field 22, guaranteeing that no maxima are missed. 2023PF00341 23 It may be intended to use the low-resolution map to quickly identify areas that might contain the global maximum and areas that are unlikely to. The aim is to minimize the volume that needs to be searched at high resolution. This involves: 1. Assessing the probability over the entire 3D search window using the low-resolution probability field 22. 2.Search for the best voxel in the low-resolution 3D space that has not yet been considered. Denote this value as Li. If Li < Hbest, terminate the procedure, where Hbest denotes the best match between lines 6 and linear object 7. 3. Evaluate the search volume within voxel i using the high-resolution probability field 22. Assume the probability of this voxel is Hi. Note that Hi ≤ Li because the low-resolution probability field 22 overestimates the probabilities. If Hi > Hbest, set Hbest = Hi. This multi-resolution strategy is not limited to just two resolutions, but can be implemented with three or more resolution granularities. Although implementing the multi-resolution approach significantly reduces runtime, the amount of memory is always determined by the granularity of the low-resolution probability field 22.Therefore, this brute-force grid approach (brute-force method 16) can be combined with the numerical optimization method 17. The granularity of the finest resolution is limited to a value (approximately 30 centimeters). This will naturally result in a ML estimate of lower accuracy. Therefore, following this coarse brute-force ML estimation, a classic numerical Gauss-Newton search for the exact ML estimate is initiated. Such numerical searches are never guaranteed to converge to the true maximum, as local maxima or even divergences can occur. However, thanks to the brute-force method 16, a close approximation to the exact solution has already been achieved. Therefore, the convergence is very good. Let Xi be the current estimate of the ML estimate, then each line starting point j is linearized around Xi using the innovation function from equation (2.18): 2023PF00341 24. Thanks to linearization, we found that J r i,jis fixed and no longer depends on X. This results in the joint probability from equation (2.17) for all line starting points 20 as: If we then assume that the cost function C is differentiable (which is true for Blake-Zisserman), and if we also linearize it to obtain all, we get: It should be noted that after linearization the values ​​C r i,j are constants and do not depend on X. The joint probability is therefore: The optimization problem can then be formulated as follows: 2023PF00341 25 From equation (2.19) we can then see: Since C r i,j are constants, equation (2.23) becomes: Now assume that C r i,j never becomes 0 or by replacing the terms that would become 0, the weighted covariance is defined as: The solution of the Gauss-Newton iteration is then simply: To model the fact that the line starting points 20 that do not extend into the minimum and maximum detection range of a particular ray 8 are not affected by the probability of such a ray 8 (since the slope of the probability outside the camera's detection range is 0), such terms are excluded from the sum in equation (2.27). This has the same effect as truncating the probability field 22 of a ray 8 outside the camera's minimum and maximum detection range, as was done in 2023PF00341 26 probability field 22 for the brute-force method 16. Finally, it is shown how the algorithm can be extended to cost functions other than Blake-Zisserman. Several examples of alternative cost functions can be found, for example, in the paper by R. Hartley and A.Zisserman entitled "Multiple View Geometry in Computer Vision" (New York, NY, USA: Cambridge University Press, 2nd ed., 2003). Instead of adding a constant clutter probability to equation (2.14) as Blake-Zisserman does, these alternative cost functions modify the squared error from equation (2.13) to prevent it from becoming too large too quickly. This is intended to limit the impact of outliers on the final result, as this is the well-known weakness of least squares estimates. The probability of a line starting point 20 from equation (2.16) is then modified for any cost function C as follows: The total probability of all line starting points 20 is then: It can already be determined that the probability field 22 must be calculated for all cells, since there is no initialization with a fixed clutter probability as in Blake-Zisserman. This makes this procedure impractical for the brute-force approach, as the runtime required to construct the probability field 22 would explode. It may still be possible to use numerical Gauss-Newton optimization, which can be started with the Blake-Zisserman brute-force estimation. However, a glance at equation (2.29) shows that the derivation will be tedious. However, an approximation can be made to simplify this. First, note that: 2023PF00341 27 Jensen's inequality is now used, since the logarithm is a convex function: Therefore, the optimization problem from equation (2.30) is redefined as: It is now about the current estimate Xi linearized, as was done before, and the Jacobian matrices J r j,j become independent of X and can therefore be neglected, so that we obtain: Thus, the same problem is obtained as in equation (2.23), and the solution is still equation (2.27). However, this is not the exact Gauss-Newton solution because Jensen's inequality was used. Nevertheless, the solution to equation (2.33) will increase equation (2.30), since equation (2.32) gives a lower bound on equation (2.30). Some examples of alternative cost functions are given below. The Cauchy cost function is given by: 2023PF00341 28 The parameter b determines the range in which the cost function closely approaches C(δ) = δ. The cost function is not convex, and b must be carefully chosen to converge to the desired minimum. An interesting and powerful function is the median cost function: the median of the distribution is given (mean absolute error, since δ is a squared error and therefore the square root of δ is the absolute error). The cost function is convex, parameter-free, and very robust. The disadvantage is that it is not continuously differentiable at 0, i.e., it is not C 1 However, the Huber cost function is C 1 and provides an approximate solution that converges to the median when b → 0: To get the property C 1 To show this, the cost function and its first-order derivative at b2 are calculated, approximating the value from the left and right. The first-order derivative is:The Huber cost function has been shown to have slightly higher accuracy than Blake-Zisserman in scenarios where there are few outliers. However, because Huber considers all inputs while Blake-Zisserman ignores very distant outliers, Blake-Zisserman is expected to perform better in densely cluttered environments. Fig. 9 illustrates the determination of the probability value, where a distance 23 between the line start point 20 and ray 8 is determined and evaluated. Alternatively, an angle 24 can be determined and assumed as the probability value, where angle 24 is the angle 24 between the line start point 20 and ray 8. The latter calculation, however, requires calculating an arctangent and is therefore more complex and computationally intensive than the first method described, which is based on determining the distance 23. In Fig.9 also shows a line extension 25 of line 6 to illustrate this angle 24. When determining the probability value and the probability field 22, a calibration error of the sensor device 2, an error of the line determination criterion 12, and / or an error of a sensor device model assumed for the sensor device 2 can be taken into account. Overall, the examples show localization using camera detection of parking lines and thus of the lines 6. When driving the vehicle 1, a plurality of linear objects 7 are detected, which are also present as lines 6 in the map 14. By comparing at least sections of the lines 6 from the map 14 and the detected linear objects 7, the position and orientation of the vehicle 1 within the map 14 can be deduced. For this purpose, distances and base points are minimized.The goal is to locate vehicle 1 within parking environment 4. This can generally be achieved by detecting parked vehicles 1, pillars, and / or walls of parking environment 4. Although this already provides valuable information for localization, it has the following disadvantages: The position and size of parked vehicles 1 change every day. This information is therefore not very accurate. However, it can still be used to some extent. The pillars are sparse and may be far apart. The walls may be far apart and may not always be detected, although long-range sensor devices 2 such as radars would solve this problem. Nevertheless, the wall at the end of a row of parking spaces 5 would not provide any information about the lateral localization of vehicle 1 (along the direction of the wall).To improve localization, the camera's line detection (sensor device 2) is used. This has the advantage that the parking space markings are part of the infrastructure (i.e., the parking environment 4) and remain fixed in one location. However, a technical problem is the fact that the camera's line detection can have comparatively poor accuracy due to camera calibration errors or uneven ground conditions. This error is not random noise, but occurs uniformly with each measurement, which would lead to a distortion of the localization.In summary, the use of parking space marking lines (linear objects 7) detected by the camera for localization within a parking space 5 was demonstrated. Line start points 20 are used as landmarks for localization and are aligned with the rays 8 corresponding to the line start points detected by the camera. Furthermore, it can be assumed that the quality of the match is estimated using an implicit measurement function to avoid the expensive arctangent calculation. Furthermore, it can be assumed that an optimized brute-force search algorithm (brute-force method 16) is used to initialize a numerical optimization method in order to limit the resolution required for the brute-force method 16 while still obtaining a highly accurate estimate thanks to the numerical optimization step performed at the end (numerical optimization method 17).

Claims

2023PF00341 31 claims 1. A method for locating a vehicle (1) within a map (14) in which at least one line (6) is drawn, the method comprising: - determining (S2) a first piece of line information (11) that at least partially describes at least one at least partially linear object (7) in an environment of the vehicle (1), by applying a line determination criterion (12) to an environmental information item (10) that describes at least part of the environment of the vehicle (1); - determining (S3) a second piece of line information (13) that at least partially describes the at least one line (6) drawn in the map (14) by evaluating the map (14); and - locating (S4) the vehicle (1) within the map (14) by performing an assignment method (15),in which the respective linear object (7) described by the determined first line information (11) is assigned precisely to the line (6) described by the determined second line information (13), which is plotted on the map (14) for the linear object (7).

2. The method according to claim 1, wherein the first line information (11) describes an angle (21) at which an object starting point of the at least one linear object (7) is oriented to a sensor device (2) of the vehicle (1) that detects the environmental information (10).

3. The method according to one of the preceding claims, wherein the second line information (13) describes a line starting point (20) of the at least one line (6) plotted on the map (14).

4. The method according to one of the preceding claims, wherein, when carrying out the assignment method (15), a brute-force method (16) is first applied,by means of which a range of relative positions and / or relative orientations of the respective linear object (7) to the respective line (6) is determined by trial and error. 2023PF00341 32 5. The method according to claim 4, wherein a distance between two tested relative positions and / or relative orientations is initially set to a first distance value and is reduced during application, in particular gradually, until a second distance value is reached.

6. The method according to claim 4 or 5, wherein, when performing the assignment method (15), after applying the brute-force method (16), a numerical optimization method (17) is applied, in particular as soon as the distance between two tested relative positions and / or relative orientations corresponds to the second distance value. 7.Method according to one of claims 4 to 6, wherein, when carrying out the assignment method (15), a probability value is determined for each tested relative position and / or relative orientation and / or is taken from a list of predefined probability values, wherein the respective probability value describes a probability with which the linear object (7) is drawn as the line (6) in the map (14).

8. Method according to claim 7 and claims 2 and 3, wherein the probability value is determined by determining and evaluating a distance between the line starting point (20) of the at least one line (6) drawn in the map (14) and a beam (8) striking the sensor device (2) at the respective angle (21) described by the first line information (11). 9.Method according to claim 7 and claims 2 and 3, wherein the probability value is determined by determining and evaluating an angle (24) between the line starting point (20) of the at least one line (6) plotted on the map (14) and a ray (8) striking the sensor device (2) at the respective angle (24) described by the first line information (11).

10. Method according to one of claims 7 to 9, wherein a calibration error of the sensor device (2) is taken into account when determining the probability value. 2023PF00341 33 Errors of the line determination criterion (12) and / or an error of a sensor device model assumed for the sensor device (2) are taken into account.

11. Method according to one of the preceding claims, wherein the at least one at least partially linear object (7) represents a parking space marking of a parking space (5) in the surroundings of the vehicle (1).

12. Method according to one of the preceding claims, wherein the at least one line (6) drawn in the map (14) represents a parking space marking of a parking space (5) in a parking environment (4).

13. Vehicle (1) designed to carry out a method according to one of the preceding claims.

14. Control device (3) for a vehicle (1), wherein the control device (3) is designed to carry out the steps of a method according to one of claims 1 to 12. 15.Computer program product comprising instructions which, when the program is executed by a computer, in particular by a control device (3), cause the computer to carry out the steps of a method according to one of claims 1 to 12.

Images