Determination of arrangement information for a vehicle

Abstract

Method for determining arrangement information for a vehicle, wherein the arrangement information comprises a position of the vehicle (10) and an orientation of the vehicle (10) with respect to a stationary coordinate system, the method comprising: - detecting a local lane arrangement in an environment of the vehicle (10) with sensors (14) of the vehicle (10), wherein the local lane arrangement indicates an arrangement of lanes in the environment of the vehicle (10) with respect to the vehicle (10), - determining a first lane arrangement as a function of the local lane arrangement and a previously determined arrangement information for the vehicle, wherein the first lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, - determining a second lane arrangement as a function of predetermined map material, wherein the second lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, - Determining a similarity information item as a function of a geometric similarity of the first lane arrangement and the second lane arrangement by determining a plurality of pair similarity information items, wherein each of the plurality of pair similarity information items is determined as a function of a geometric similarity of a respective pair of two lanes, wherein a respective pair of two lanes each comprises a lane from the first lane arrangement and a lane from the second lane arrangement, wherein the plurality of pair similarity information items comprise a respective pair similarity information item for each combination of a lane from the first lane arrangement and a lane from the second lane arrangement, and - updating the previously determined arrangement information for the vehicle (10) depending on the similarity information.

Description

[0001] The present invention relates to a method for determining vehicle positioning information, in particular determining a vehicle's positioning by taking into account lane-accurate map information while simultaneously improving an assumption or hypothesis regarding the lane arrangement. The present invention can be used in particular in conjunction with driver assistance systems that enable or support automatic guidance of the vehicle in a longitudinal and / or transverse direction.

[0002] To provide automated driving functions for driver assistance systems in vehicles, such as passenger cars or trucks, comprehensive knowledge of the vehicle's current environment is required. An important aspect of this environment is the road infrastructure in the immediate vicinity of the vehicle, in which the vehicle's automatic driving behavior must be planned. Various types of information sources can be used to determine the most likely road infrastructure, such as prior knowledge from lane-accurate road maps in combination with a global positioning system (GPS), as well as sensor-based lane perception, for example using cameras. One problem is that all data sources, such as the map, global positioning, and lane perception, can be subject to topological and geometric uncertainties and measurement errors of various kinds.To obtain a consistent and consistent picture of the environment, the most probable position and orientation relative to the map and the most probable lanes must be determined, taking into account all data sources and their errors.

[0003] In this context, DE 10 2011 120 497 A1 discloses a system for precise vehicle position determination within a lane. The method is carried out by a vehicle having an on-board computer, vehicle sensors, a satellite positioning unit, and a database with a map at the lane level in order to determine a new position of the vehicle using map matching. In the method, new data is received from at least one of the vehicle sensors and measured values ​​are collected from the vehicle sensors. The on-board computer calculates the propagation of the vehicle position with respect to successive points in time. Furthermore, the on-board computer performs a curve fitting process in which, for example, GPS coordinates of the vehicle are obtained and the location of the vehicle within the map at the lane level is identified.The on-board computer executes a tracking program that includes using a probability distribution to update the vehicle's posture with respect to data particles, and executes a particle filtering program based on the data particles to calculate the new vehicle posture. Based on the results of the curve-fitting process, an observation model is updated. Data is read from the map at the lane level, which identifies at least one registered lane line near the vehicle, and a position of the lane line relative to a host vehicle reference system is calculated. Using range analysis, the detected and registered lane stripes and boundaries are compared, and using the compared data, an observation probability distribution is calculated with respect to image recognition data from a camera of the vehicle's sensors.When calculating a position of the lane line with respect to a reference system of the host vehicle, the lane stripes obtained from a map coordinate system at the lane level are projected onto the reference system of the vehicle and the detected and registered lane stripes and boundaries are compared, using a Euclidean type of distance analysis as the distance analysis.

[0004] DE 10 2010 033 729 B4 relates to a method for determining the position of a vehicle on a roadway. In the method, the position of the vehicle is determined with a first accuracy based on outputs from a satellite signal sensor. Data on a first vehicle environment for the position determined with the first accuracy are obtained from a digital map. Furthermore, data on lane markings in a second vehicle environment are obtained using a line detection sensor. Data on the vehicle's own motion are obtained using a vehicle dynamics sensor, and data on objects in a third vehicle environment are obtained using an environment sensor.The position of the vehicle on the roadway is determined with at least lane-specific accuracy by combining the data on the first vehicle environment from the digital map and the data on the lane markings in the second vehicle environment with the data on the vehicle's own movement or the data on objects in the third vehicle environment depending on the roadway situation.

[0005] When combining information from map data, positioning (e.g., GPS), and lane perception, it is usually assumed that some of the data is error-free and free of uncertainty. However, this is generally not the case, so this assumption leads to an inaccurate determination of the vehicle's position and / or orientation relative to the actual roadway. However, such inaccuracies are undesirable, especially in driver assistance systems that automatically steer the vehicle along the roadway, especially at close range.

[0006] DE 10 2010 005 293 A1 relates to a method for estimating a projected trajectory for a vehicle on a road, comprising monitoring a plurality of sensor inputs, determining a road geometry in front of the vehicle based on the monitored sensor inputs, determining a vehicle position with respect to the road geometry based on the monitored sensor inputs, determining a plurality of particle points in front of the vehicle representing a potential trajectory from the road geometry and the vehicle position, and using iteratively determined particle points of the plurality of particle points for navigation of the vehicle, including omitting particle points of a plurality of the particle points passed by the vehicle.

[0007] DE 10 2006 040 334 A1 relates to a method for lane detection with a driver assistance system of a vehicle comprising a sensor system for lane recognition.

[0008] DE 10 2006 040 333 A1 also shows a method for lane detection with a driver assistance system of a vehicle.

[0009] The object of the present invention is therefore to enable a more reliable determination of the position and orientation of the vehicle in relation to the actual roadway.

[0010] This object is achieved according to the present invention by a method for determining arrangement information for a vehicle according to claim 1, an arrangement determining device according to claim 11, and a vehicle according to claim 13. The dependent claims define preferred and advantageous embodiments of the invention.

[0011] According to the present invention, a method for determining location information for a vehicle is provided. The term "location information" refers to a position of the vehicle and an orientation of the vehicle with respect to a stationary coordinate system, such as a global coordinate system spanning the world, such as the World Geodetic System (abbreviated WGS). However, the stationary coordinate system can also comprise any other coordinate system outside the vehicle, such as a country-specific coordinate system. In the context of the present invention, the term "location information" is also referred to as "pose" and comprises at least a two-dimensional position in the stationary coordinate system and an orientation of the vehicle with respect to the stationary coordinate system.In the method, a local lane arrangement in the vehicle's surroundings is detected using vehicle sensors. The lane arrangement can be detected, for example, using vehicle cameras. The local lane arrangement indicates an arrangement of lanes in the vehicle's surroundings with respect to the vehicle. A first lane arrangement is determined depending on the local lane arrangement and previously determined arrangement information for the vehicle. The first lane arrangement indicates an arrangement of lanes in the vehicle's surroundings with respect to the stationary coordinate system. In other words, a (first) lane arrangement with respect to the stationary coordinate system is determined from the local lane arrangement, which results from the vehicle's view or perception.Based on predefined map material, a second lane layout is determined for the current surroundings of the vehicle with respect to the stationary coordinate system. Based on a geometric similarity between the first lane layout and the second lane layout, similarity information is determined, and the previously determined layout information for the vehicle is updated based on the similarity information. For example, shifts may be necessary to map the first lane layout to the second lane layout. Based on these shifts, the layout information or pose for the vehicle can be updated.By determining the geometric similarity between the lane layout detected in the vehicle's surroundings and the map-based lane layout, sensor-based and map-based lane data can equally contribute to updating the layout information for the vehicle.

[0012] In one embodiment, a previously determined lane arrangement hypothesis is further updated depending on the similarity information. The lane arrangement hypothesis specifies a hypothesis for an arrangement of lanes in the surroundings of the vehicle with respect to the stationary coordinate system. Thus, based on the geometric similarity of the first and second lane arrangements, both the arrangement information, i.e., a pose estimate for the vehicle, and a lane arrangement hypothesis can be jointly updated and optimized.

[0013] In a further embodiment, the vehicle arrangement information comprises parameters of a multidimensional normal distribution for the position and orientation of the vehicle. For example, the position can comprise a two-dimensional normal distribution for the position of the vehicle in a Cartesian coordinate system, and the orientation can comprise another normal distribution for a rotation angle of the vehicle relative to a predetermined orientation of the coordinate system. Coordinates of the lane arrangement hypothesis, for example, points along a lane center, can also be represented as multidimensional normal distributions. The parameters of each of these normal distributions can include, for example, an expected value and a variance.By basing the layout information and / or the lane layout hypothesis on a normal distribution-based representation approach, errors and inaccuracies in the associated acquisition of input data can be modeled and accounted for. In particular, errors or inaccuracies in the map-based lane layout can also be considered. Furthermore, all sensor-based input data can be acquired using cost-effective sensors, since the respective uncertainties can be directly incorporated into the calculation using normal distributions.

[0014] Preferably, a lane of a lane arrangement is represented by a point list representing the lane center. Each point of the point list has parameters of a multidimensional normal distribution for a position relative to the stationary coordinate system. Map material comprising point lists marking the center of lanes is commonly available and can therefore be advantageously used in the method. By representing each point of the point list as a normal distribution, uncertainties in the map material can be taken into account. For example, the coordinates specified by the map material in each dimension can be used as the expected values ​​of the normal distribution in the respective dimension. A standard deviation of the normal distribution in the respective dimension can be selected, for example, based on the map accuracy.By connecting two or more consecutive points in the point list, a desired orientation of a vehicle within a lane can be determined. This allows the second lane arrangement, determined based on the given map material, to contribute to both updating the position and the orientation of the vehicle.

[0015] In a further embodiment, the previously determined arrangement information for the vehicle is updated depending on odometry data of the vehicle. The odometry data of the vehicle can, for example, include information from wheel sensors, steering angle sensors, or acceleration sensors of the vehicle. Using the odometry data, a relative change in the arrangement of the vehicle with respect to a previous position can be determined. Based on the previously determined arrangement information, for example, current arrangement information can be determined by incorporating the odometry data, wherein inaccuracies in the previously determined arrangement information and measurement inaccuracies of the odometry data can be taken into account in the form of normal distributions. This allows the odometry data of the vehicle, including its possible errors, to be taken into account when updating the arrangement information for the vehicle.

[0016] In a further embodiment, a stationary position of the vehicle is detected. The stationary position indicates a position of the vehicle relative to the stationary coordinate system. The stationary coordinate system can be, for example, a global coordinate system, wherein the stationary position is detected, for example, using a satellite positioning system. The previously determined arrangement information for the vehicle is updated depending on the detected stationary position of the vehicle. Since the detected stationary position of the vehicle may have a measurement inaccuracy, the stationary position of the vehicle in the form of a multidimensional normal distribution can be taken into account when updating the previously determined arrangement information.

[0017] Using the previously described normal distribution-based representation approach for the vehicle's layout information and the additional information for updating the vehicle's layout information, such as odometry data, map information, camera-based environmental information, and satellite-based positioning information, this information, which is subject to measurement errors, including its potential errors, can be easily processed to update the vehicle's layout information. In particular, the normal distribution-based representation approach allows the layout information to be updated within a few milliseconds on a standard computing system, such as a microprocessor controller in a vehicle, and is thus suitable for real-time computing in the vehicle.A repetition rate for repeatedly performing the procedure can, for example, be in a range of 25 to 400 repetitions per second.

[0018] In a further embodiment, additional features in the vehicle's surroundings can be compared with corresponding features of the map material to determine the similarity information. For this purpose, lane information for the first lane arrangement is determined using the vehicle's sensors. The lane information includes, for example, lane marking types, stop line positions, or traffic sign positions. Lane marking types can include, for example, a solid lane marking, a broken lane marking, a double solid line, and the like. The lane information for the first lane arrangement is compared with map material-based lane information for the second lane arrangement. This can improve the assignment of the lanes detected by the vehicle's sensors to lanes from the map material.

[0019] The first lane arrangement comprises an arrangement of at least two lanes, and the second lane arrangement comprises an arrangement of at least two lanes. To determine the similarity information, a plurality of pair similarity information items is determined. Each of the plurality of pair similarity information items is determined depending on a geometric similarity of a respective pair of two lanes. Each respective pair of two lanes comprises one lane from the first lane arrangement and one lane from the second lane arrangement. In other words, one pair similarity information item is determined for each combination of one lane from the first lane arrangement and one lane from the second lane arrangement.If, for example, the first lane arrangement comprises two lanes and the second lane arrangement also comprises two lanes, four pair similarity information items are determined, namely a first item for a similarity between a first lane of the first lane arrangement and a first lane of the second lane arrangement, a second item of pair similarity information items for a geometric similarity between the first lane of the first lane arrangement and the second lane of the second lane arrangement, a third item of pair similarity information items for a geometric similarity between a second lane of the first lane arrangement and the first lane of the second lane arrangement, and a fourth item of pair similarity information items for a geometric similarity between the second lane of the first lane arrangement and the second lane of the second lane arrangement.By processing any number of lanes, the vehicle layout information and the lane layout hypothesis can be updated even in complex scenarios, such as intersections or exits. For example, the previously determined vehicle layout information can be updated depending on the similarity information by determining a respective weighted update information for each of the multiple pair similarity information items, depending on the respective pair similarity information. The weighted update information includes vehicle layout information assuming that the lanes of the respective pair relate to the same actual lane.The weight of the weighted update information indicates the probability that the lanes of the respective pair relate to the same actual lane. The previously determined arrangement information is updated depending on the weighted update information, for example, by multiplying an update based on pair similarity information by the weight. The vehicle's arrangement information and the weighted update information can each include parameters of a multidimensional normal distribution for the position and orientation of the vehicle. To update the previously determined arrangement information for the vehicle, the normal distribution of the arrangement information is multiplied by the normal distributions of the update information.This makes it easy to update the vehicle's layout information even in complex situations and scenarios, such as multi-lane roads, intersections and exits.

[0020] According to the present invention, a layout determination device for determining layout information for a vehicle is further provided. The layout information includes a position of the vehicle and an orientation of the vehicle in position relative to a stationary or global coordinate system. The layout determination device includes sensors for detecting a local lane layout in the surroundings of the vehicle. The sensors can, for example, include cameras of the vehicle. The local lane layout indicates an arrangement of lanes in the surroundings of the vehicle with respect to the vehicle and can, for example, be determined from image data of the camera, optical properties of the camera, and a layout of the camera using suitable image processing.The arrangement determination device further comprises a processing device capable of determining a first lane arrangement depending on the local lane arrangement and previously determined arrangement information for the vehicle. The first lane arrangement indicates an arrangement of lanes in the surroundings of the vehicle with respect to the stationary coordinate system. In other words, the processing device converts the local lane arrangement into a corresponding global lane arrangement, taking into account the most recently determined arrangement information or pose for the vehicle. The processing device is further capable of determining a second lane arrangement depending on predetermined map material. The second lane arrangement indicates an arrangement of lanes in the surroundings of the vehicle with respect to the stationary coordinate system.The second lane arrangement can be determined, for example, using the previously determined arrangement information for the vehicle from the specified map material. Finally, the processing device determines similarity information based on a geometric similarity between the first lane arrangement and the second lane arrangement. This similarity information is used to update the previously determined arrangement information for the vehicle. Thus, the arrangement determination device is suitable for implementing the previously described method or one of its embodiments and therefore also includes the advantages described in connection with the method.

[0021] The present invention further relates to a vehicle comprising the previously described arrangement determination device and a driver assistance system. The driver assistance system provides an automatic driving function for the vehicle, for example, automatic longitudinal and lateral control of the vehicle. The driver assistance system is coupled to the arrangement determination device, and the automatic driving function is executed based on the updated arrangement information from the arrangement determination device. The vehicle therefore also includes the advantages previously described in connection with the method and its embodiments.

[0022] As previously described, a previously determined lane arrangement hypothesis can be updated depending on the similarity information. The lane arrangement hypothesis specifies a hypothesis for an arrangement of lanes in the surroundings of the vehicle with respect to the stationary coordinate system. The updated lane arrangement hypothesis can be taken into account by the driver assistance system when executing the automatic driving function, for example, by relating the arrangement information or pose of the vehicle to the updated lane arrangement hypothesis and, in particular, controlling lateral control of the vehicle on this basis.

[0023] Although the above-described embodiments of the present invention have been described independently of one another, it is clear that these embodiments can be combined with one another as desired.

[0024] The present invention will be described in detail below with reference to the drawings. Fig. 1 schematically shows a vehicle according to an embodiment of the present invention. Fig. 2 shows steps of a method for determining arrangement information for a vehicle according to an embodiment of the present invention. Fig. 3 shows an example of a graphical representation of a factor graph that can be used to determine arrangement information for a vehicle according to an embodiment. Fig. 4 shows a schematic representation of an update cycle for a lane hypothesis and a pose hypothesis according to an embodiment of the present invention. Fig. 5 shows a factor graph for a lane assignment according to an embodiment of the present invention. Fig.Figure 6 shows an average positioning error for different sources and sizes of disturbances. Fig. Figure 7 schematically shows map-based lane arrangements, locally recorded lane arrangements and poses for a vehicle calculated from them.

[0025] Fig.1 shows a vehicle 10 with a layout determination device 11 and a driver assistance system 12. The driver assistance system 12 is capable of performing an automatic driving function for the vehicle 10. To do so, the driver assistance system 12 requires information about the lane the vehicle 10 is currently in and the current position of the vehicle 10 in relation to the lane. The layout determination device 11 provides this information via a connection 13. The layout determination device 11 comprises a sensor 14, for example one or more cameras, for detecting a local lane layout in an environment of the vehicle 10. The local lane layout can be calculated, for example, by a processing device 15 of the layout determination device 11 using suitable image processing, taking into account the arrangement of the camera 14 on the vehicle 10.The processing device 15 is further equipped with a global positioning system, which, for example, receives signals from satellites via an antenna 16 and calculates a global position of the vehicle 10 therefrom. The processing device 15 can further be connected via a vehicle bus 17 to other components of the vehicle 10 that provide odometry data of the vehicle. For example, the processing device 15 can receive information from a wheel sensor 18 to determine a movement of the vehicle 10. Alternatively or additionally, further wheel sensors, acceleration sensors, or a steering angle sensor can be coupled to the processing device 15 to calculate odometry data of the vehicle 10.

[0026] To provide an automatic driving function from the driver assistance system 12, the driver assistance system 12 requires comprehensive knowledge of the current surroundings of the vehicle 10 as well as the vehicle's own position and orientation within the current surroundings. An important aspect of this surroundings is the road infrastructure in the immediate vicinity of the vehicle 10, since the automatic behavior of the vehicle, for example, longitudinal and lateral control of the vehicle, must be planned in this area. To determine the most likely road infrastructure, the layout determination device 11 uses various types of information sources: prior knowledge from lane-accurate road maps in combination with global positioning via, for example, GPS, as well as sensor-based perception of the lanes via, for example, the camera 14.To obtain a consistent and internally congruent image of the surroundings, the most probable pose—that is, the position and orientation of the vehicle 10—relative to information from the map, as well as the most probable lane, are determined, taking into account all data sources and their respective errors. For this purpose, the positioning device 11 uses the algorithm shown in . Fig. 2 procedures described in detail.

[0027] As input data, the arrangement determination device 11 uses a global position measurement, a lane-accurate digital road map and a so-called perception method which generates lane hypotheses from sensor data, in particular from the camera 14.

[0028] Global position measurement can be performed, for example, using a global positioning system such as GPS or Galileo. The measurement method determines a two-dimensional position, for example, in a global reference system of geodesy, such as the World Geodetic System (WGS). Alternatively, the two-dimensional position can be determined in any other stationary, i.e., non-vehicle-related, reference system. A measurement variance is or is assigned to the measurement method of global position measurement. Furthermore, the global orientation of the vehicle, including a corresponding measurement variance, is determined using the global position measurement, for example, using a compass. The two-dimensional position in conjunction with the orientation of the vehicle will hereinafter be referred to as vehicle arrangement information or vehicle pose.

[0029] The lane-accurate digital road map provides information for individual lanes. The lanes are defined as a list of points, each with a position including the modeling variance. The point list represents the lane center, and all points on the connecting lines are interpreted as linear interpolations of the neighboring modeled points. Furthermore, it can be expanded with additional features, such as the types of lane markings on the right and left lane edges or the positions of stop lines and traffic signs.

[0030] The perception process generates lane hypotheses from sensor data from the vehicle's sensors. The sensors used and the methods for determining the lane hypotheses are irrelevant. However, in particular, cameras on the vehicle can be used, for example, and the lane hypotheses can be determined based on digital image processing. The result of the process is the lane hypotheses as point lists, each representing the lane center. Since the perception process is carried out using vehicle sensors, the points in the point lists are located in a coordinate system aligned with the vehicle, a so-called local reference system, known as "ego-local coordinates." The coordinates of the points also exhibit a measurement variance.As an extension, the same additional features as in the map can be extracted from the sensor data, such as lane marking types, stop lines and traffic sign positions.

[0031] Furthermore, odometry data of the vehicle are used as input data for determining the arrangement information, which describe the vehicle's own movement continuously or between certain points in time, so-called time steps.

[0032] The Fig.The method 20 shown in Figure 2 comprises method steps 21 to 28. Initialization takes place in step 21. For example, pose hypotheses are determined at a current GPS position, map material is loaded, and a current lane hypothesis is determined from the map material for the current GPS position. The subsequent steps 22 to 28 are executed cyclically, with a cycle lasting a few milliseconds, for example, 3 to 40 ms. The cycle duration can, for example, correspond to a duration required for acquiring sensor data from, for example, the camera 14, the wheel sensors 18, and the global positioning system.

[0033] Since method 20 is intended to account for measurement errors and inaccuracies in the map information and the perceived environment, the pose and lane hypotheses are represented as multidimensional normal distributions (Gaussian mixture distribution). Likewise, information for updating the pose and lane hypotheses, such as results from the global position measurement, the perception process, and the odometry data, as well as information from the digital road map, are represented as corresponding multidimensional normal distributions. Updates can then be implemented, for example, by multiplying all components of these distributions. In this way, in step 22, the pose hypothesis from the last time step or from the initialization is updated using the odometry data and thus transferred to the current processing time.In step 23, the pose hypothesis is updated using the global position measurement. In step 24, the local lane arrangement is detected using the perception method described above, and in step 25, these locally detected lanes are transferred to the global or stationary coordinate system using the current pose hypothesis. This provides a first lane arrangement that was formed based on the locally detected lanes. For the current pose hypothesis of vehicle 10, a second lane arrangement is determined from the map material. The sensor-based lanes from the first lane arrangement are then compared pairwise with the map-based lanes from the second lane arrangement (step 26). For this purpose, a geometric similarity of each lane pair is determined, including for systematic shifts of the lanes relative to one another.The similarity value can be enriched and improved by comparing additional information, such as lane marking types or stop line and traffic sign positions. Assuming that the respective lane pair represents one and the same actual lane, a respective weighted pose update is created. The pose results from the various displacements relative to each other, and the weight is determined by the similarities between the lanes. In step 27, the current pose hypothesis is updated using all pose updates obtained in this way. The result is a new pose hypothesis, which can be used in the next cycle of the process. Additionally, the pose hypothesis can be output to the driver assistance system 12.In step 28, the lane hypothesis of the initialization or the last cycle of the method is updated using the sensor-based lanes transferred to the global coordinate system. Additionally, the results of the lane data comparison from step 26 are incorporated into the lane hypothesis update. The updated lane hypothesis can also be output to the driver assistance system 12, so that the driver assistance system 12 can control the vehicle 10, particularly in the near range, based on the pose hypothesis and the lane hypothesis. The lane hypothesis can be expanded from map data to the desired perception horizon and made available to the driver assistance system 12.

[0034] In method 20, pose determination and the updating of sensor-based lane data are performed in parallel with map data and thus jointly optimized. The method works with any number of lanes and is therefore capable of covering even complex scenarios such as intersections and exits. All sensor-based input data can be based on inexpensive sensors, since the respective uncertainties can be incorporated into the calculation using normal distributions. Likewise, relatively simple map data can be used, so that, for example, no georeferenced sensor features need to be directly available in the map. The normal distribution-based representation allows the entire cycle to be calculated within a few milliseconds on a conventional microprocessor or on a conventional computing unit in a vehicle and can therefore be used for real-time calculations in the vehicle.

[0035] An embodiment of the above-described method using a factor graph-based approach will be described in detail below.

[0036] The method performs a rough localization of the vehicle using low-cost sensors, such as GPS sensors and camera-based lane detection, and maps with lane-level accuracy, with the perceived road network and position determination being updated simultaneously. A sum-product algorithm adapted to a factor graph is applied, which models the dependencies between observed and hidden variables. The sum-product algorithm is also called "belief propagation" or "sum-product message passing." Belief propagation can be used in conjunction with acyclic graphs or general graphs. If the graphs contain cycles or loops, the algorithm is also called "loopy belief propagation."Belief propagation is one of the so-called message passing algorithms that can be applied to factor graphs. Message passing within the graph relies on multimodal normal distributions for variable representation and quadratic noise models, resulting in a fast and well-defined computational structure. Simulations show that positioning accuracy is insensitive to most types of measurement noise, except for a constant offset from global pose measurements, which can nevertheless be reduced by a factor of 8. Tests in real-world environments with an average positioning error of 1.71 m in an urban scenario demonstrate the applicability of the approach for automated driving tasks, as well as its real-time capability, with an average execution time of 3 ms on a typical computer system.

[0037] The trend in driver assistance systems (DAS) is moving toward assisted or even guided automated driving. Such tasks require a reliable understanding of the current environment, particularly the detection of drivable areas and the geometric and topological structure of the road and its lane network.

[0038] In such driving tasks, humans use their visual perception and prior knowledge of typical situations in combination with additional information sources, such as road maps, to improve their understanding of a scene. Automated systems use the same principles, using sensor inputs from, for example, cameras or range sensors, as well as static sources, such as maps, in conjunction with global positioning systems. However, these systems suffer from noisy and erroneous input data. Lane detection based on sensor input can be confounded by geometric noise, miss lanes altogether, or detect non-existent lanes, i.e., be convinced of false results, leading to topological ambiguity that increases at least linearly with distance.Road map data can provide a topological structure with lane-level accuracy and infinite range. However, their production can be subject to geometric inaccuracies and even further topological errors, for example, due to changes in the actual road network. In addition, the required global positioning system typically depends on external inputs, such as satellites, and is therefore only accurate to a certain extent.

[0039] In the embodiment described below, noisy lane hypotheses from sensor inputs, noisy road maps with lane-level accuracy, and noisy global positioning information are used to simultaneously improve the accuracy of all three. This is achieved by collecting all measured values ​​and associations between sensor data and road map data as constraints in a specially designed factor graph, whose optimal parameters are determined by applying loopy belief propagation.

[0040] In the following, a theoretical background of factor graphs and loopy belief propagation is given, as far as it is necessary for a comprehensive understanding of the embodiment described subsequently.

[0041] Factor graphs are graphical representations used for groups of probability variables and their conditional dependencies. A factor graph decomposes a connected probability distribution P(x1, x2,..., x n ) in factors ψ i only depending on a subset of random variables, as shown in equation (1), where Z is a normalizing constant. P(x1,x2,⋯,xn)=1Z∏iψi(xψi1,xψi2,⋯,xψim)

[0042] Factor graphs are usually represented as unidirectional graphs with circular nodes as variables and rectangular nodes as factors, which are connected by edges for each variable used in the factor, as in Fig. 3 is shown.

[0043] A common task is to determine the marginal distribution of hidden variables X h taking into account the given values ​​of known observed variables Xo The factor graph can be reduced to the extent that it contains the conditional probability P(X h |X o ) by removing all observed variables from the graph and setting all occurrences of the variable in the factors to the observed value. These observations can be distributed using a message passing algorithm, such as the sum-product algorithm, using the dependencies of the factors by recursively passing information, called messages, i→i according to equation (2). mi→j=∫x∈Si\Sjψi(x)∏k∈Ni\{j}mk→idx

[0044] An outgoing message from node i to node j is the product of all incoming messages from all neighbors N iexcept for j times the associated factor, which is applied to the subset of variables S used by node j j is marginalized.

[0045] For variable nodes, ψ i be assumed to be 1. The final marginal distribution of a variable is the product of all incoming messages.

[0046] Since incoming messages must be known in advance, message processing must be timed to satisfy this precondition. Tree-like factor graphs guarantee such a timing that finds the globally correct solution. However, cyclic graphs do not exhibit valid scheduling, since all messages in a cycle require input from another message in the cycle. Nevertheless, empirical studies have shown that the sum-product algorithm can find locally optimal solutions if all updates are repeated until convergence. In this case, messages can be initialized with a uniform distribution. The resulting algorithm is then called "loopy belief propagation."

[0047] In the embodiment described below, a factor graph is used for a simultaneous localization and mapping (SLAM) technique to determine lanes of a road network in general for arbitrary curves and the vehicle's position within them. To achieve this, the following two tasks must be considered. First, the problem of representing the current probability distribution over the existing features (i.e., poses and lanes) with methods for incorporating new measurements into these probabilities. And second, the mapping problem between measurements of identical objects in the real world, i.e., lanes observed at different times and lanes from a road map. The mapping problem and measurement updates of variables are often solved separately.However, they can be solved simultaneously by considering several possible assignments at once. The general principle is described in . Fig. 4, in which a current pose and current lane hypotheses are combined with new lane observations by determining appropriate mappings and updating both the lane geometry and pose based on these mapped lanes. Additionally, global pose measurements provided by global positioning systems and odometry measurements provided by on-board measurement units are considered.

[0048] Fig.Figure 4 schematically shows the update cycle. Existing multimodal pose and lane hypotheses are combined with new lane observations by searching for similar lanes and using these mappings to update the lanes and poses.

[0049] Both problems are modeled as a single factor graph that includes all additional input sources using the variables defined in Table 1. symbol Description Crowd q t Global pose measurement at time t P o o t Odometry measurement from t -1 to t P o y i i -th lane observed from sensor data in ego local coordinates C o m j j -th lane from the map material inglobal coordinates C o P t Actual global pose at time t P h l i Actual lane, which corresponds to the lane i-th observed from sensor data or the lane from the map material in global coordinates C h S i] Indicator of whether l i and l j are the same {0, 1} h

[0050] All pose-related variables are in the set P = R 2 ∩ R [-π,π] a 2D position with an orientation or orientation. All curve-related variables are defined as color stripe centerlines in C, where the scope of all functions maps a range parameter z ∈ [z1, z2] to a pose in P using a natural parameterization, i.e., the arc length of the curve γ between γ(z i ) and γ(z j) is |z i - z j |. The variables in the table are marked as monitored (o) or hidden (h). Note that the number of variables in the factor graph must be known in advance. Therefore, a separate variable is used for the corresponding actual lanes corresponding to each lane observation, since their total number can be predetermined or chosen to be sufficiently large. The true number of actual lanes is unknown and can only be implicitly inferred from the cue variables for each pair of lanes.

[0051] Fig.Figure 5 shows a corresponding factor graph. In the factor graph, these variables are connected to factors that describe necessary preconditions and assumed conditional dependencies. The left part of the graph solves pose tracking and is a Markov process in the factor graph representation. The middle part ensures the updating of lanes, for example, curves, while the right part encompasses the assignment problem. The graph expands to the entire number of time steps and lane observations accordingly. In total, there are five different types of factors that model a type of observation error: 1) Factor ψ1 (see equation (3)) connects the current pose p t with the observed pose q t assuming a normally distributed measurement noise with a covariance Σ qt , where N [µ,Σ](x) the value of a multidimensional normal distribution function at point x with mean µ and covariance matrix Σ qt , is. ψ1(pt,qt)=N[qt,∑qt](pt) 2) The factor ψ2 (see equation (4)) connects the current poses of two consecutive time steps with an odometry measurement o t , where again a normally distributed measurement noise with a covariance Σ ot is accepted. ψ2(pt,pt+1,ot)=N[ot,∑ot](pt−pt−1) 3) The factor ψ3 (see equation (5)) connects a lane from the map material m i with the actual lane l i , which corresponds to this map lane, where normally distributed modeling inaccuracies with a covariance Σ mfor all points of the map. It is assumed that the curves use the same parameter z, which creates a uniform mapping between curve points. The factor is determined by the length ||Z|| of the connected area Z = def mi^li normalized to obtain comparable values. Factors of curves without a connecting region are set to zero. ψ3(li,mi)=1‖Z‖∫z∈ZN[mi(z),∑m](li(z))dz 4) The factor ψ4 (see equation (6)) connects a lane observation of sensor data y i with the actual lane l i , which corresponds to this observation and the current pose at the time of observation. Again, the difference between the measurement and the actual lane is calculated with a normally distributed observation noise ∑yi(z) which is given individually for each point of the observed curve (parameterized with z). Accordingly, the factor is given by the length ||Z|| of the connected region Z = def yi^li normalized. ψ4(li,yi,pt)=1‖Z‖∫z∈ZN[yi(z),∑yi(z)](li(z)−pt)dz 5) The factor ψ5 (see equation (7)) connects pairs of actual lanes (l i , l j ) with their binary display variables s ij The small constant value α controls the similarity threshold with which two lanes are marked as equal or unequal (0 < α < 1). The indicator function I(cond.) takes the value 1 if the condition is true and the value 0 otherwise. Again, ||Z|| is the length of the connected region Z = def li^lj . ψ5(li,lj,sij)=|(sij=0)α+ |(sij=1)1‖Z‖∫c∈Z|(li(z)=lj(z))dz

[0052] Because the proposed factor graph contains cycles, a message-passing schedule is necessary to run the loopy belief propagation algorithm. Since the graph contains time-dependent variables that are inserted during real-time computations, information can only be passed into the future and never back to the past to ensure a constant runtime of the message-passing cycle. However, the time-independent variables can be updated with new information at each time step. Therefore, their messages are recalculated once at each time step. Although multiple message-passing cycles per time step can lead to faster convergence, it is preferable to do this only once, as slow convergence improves the ability to find a globally optimal solution.Taking into account the guidelines mentioned, the chronological sequence indicated by the arrows and their numbering in the . Fig. 5 to execute the loopy belief propagation algorithm, which ultimately converges to a local optimum of the joint probability of all hidden variables.

[0053] Applying the method to real-world problems requires a suitable representation of all variables and methods for calculating the messages of the factor graph using these variables. Three different types of distributions of the variables must be represented: poses in the set P, curves or lanes in the set C, and the binary indicator variables.

[0054] a) Pose variables and messages: Pose distributions of p are calculated as a weighted mixture of normal distributions with a mean pose µ i and a covariance matrix Σi for the position and angle, which is the form ∑iwiN[μi,∑i](p) Since it can be assumed that all angular variances are relatively small, the calculations are simplified by approximating the enveloping angular distributions with a simple normal distribution.

[0055] The general message computation (see equation (2)) requires a multiplication and marginalization of these distributions. Using the property of all normal distributions that ∫cN[μ,∑](x)dx=c holds, marginalization can be easily achieved and the product of two pose distributions can be calculated using equation (8). ∑iwiN[μi,∑i](p)∑jwjN[μj,∑j](p)=∑kwkN[μk,∑k](p)with ∑k−1=∑i−1+∑j−1;μk=∑k(∑i−1μi+∑j−1μj)wk=wiwjN[μj,∑i+∑j](μi)

[0056] To avoid an increasing number of mixing components, similar components can be approximated by a single distribution using equation (9). wiN[μi,∑i](p)+wjN[μj,∑j](p)≈wkN[μk,∑k](p)with wk=wi+wj;μk=1wk(wiμi+wjμj)∑k=1wk(wi∑i+wj∑j+w1w2(μi−μj)(μi−μj)T)

[0057] b) Curve variables and messages: Curve distributions are represented as a polygonal chain with individual pose distributions γ(z) ∈ P for each point z ∈ Z, where Z is the group of represented points. The pose distributions between them are assumed to be a linear interpolation of the neighboring points. Thus, additional curve points can be inserted as needed, and the total number of points per curve can be reduced by removing those points that are already similar to the linear interpolation of their neighboring points.

[0058] The curve parameter z is considered the same for all message calculations involving multiple curves. To find this common parameter z of two curves, the offset between the original parameters z1 and z2 of these curves is determined by projecting all points of both curves onto each other and taking the average parameter difference between the original points and their projection onto the other curve as the parameter offset. Using this assumption, two further simplifications can be applied: 1) All curve-related messages can be calculated separately for each z defined in the curves concerned. 2) The integrals in the factors ψ3, ψ4 and ψ5 are then reduced to their integrand, since the common domain is only the single z for which the message is currently computed.

[0059] It should be noted that these simplifications do not apply to the Fig. 5, the message labeled 10 can be applied because the curves involved are in different coordinate systems. Thus, the integral must be evaluated for each possible pair of curve parameters, resulting in a complete set of weighted pose distributions.

[0060] c) Binary displays: These are represented with a probability value in the range [0,1], which allows the display functions in ψ5 to be represented by the continuous probabilities s ij or 1 - s ij to replace.

[0061] In summary, the presented procedures are sufficient to calculate all messages using the general formula of equation (2), maintaining the chosen representation of the variables. Furthermore, the calculations are computationally efficient. The two main reasons for this efficiency are, first, that all factors use only quadratic noise, allowing a coherent representation largely based on normal distributions, and second, that the mentioned approximations can be used to limit the total number of mixing components or points per lane to a constant minimum, ensuring a limited maximum runtime per update cycle.

[0062] The previously described embodiment of the method was evaluated in two ways: on simulated test data and in a real scenario.

[0063] Simulating precise inputs and adding artificial noise enables systematic tests that reveal sensitivity to different types of sensor noise, which would not be possible in real-world tests. The applied test environment creates continuous roads with random curvatures, random curvature changes, and random lane counts, with random lane splits or merges, as well as intersections with random angles to the main travel direction. The current pose is simulated by driving along these lanes with random lane changes in between. The simulated scenario covers most situations encountered in rural or urban environments. Different types of noise were added to the input data used in the procedure. The global position measurement exhibits a normally distributed offset. N[uq,σq2](q) and the global angle measurement has an offset N[uα,σα2](q) The lane measurements are in their field of vision l y limited and random geometric inaccuracies, for example a lateral, angular or curved offset, are calculated with standard deviations σ l . σ a , or σ c , added. The lanes are randomly assigned with probability y - hidden or with probability y + added.

[0064] Each noise source is evaluated separately by simulating a 100 km journey. Fig. Figure 6 shows the average positioning error for each noise source and magnitude. Fig. 6 clearly shows that geometric inaccuracies of the lanes (l y , σ l , σ a , σ c , y - , y +) and white noise in measurements of the global position and angle (σ q , σ a ) have only minor effects on the positioning accuracy with a maximum increase of 0.21 m compared to perfect data. Systematic shifts in the global position and angle measurements have a larger effect on the error, but the algorithm is still able to reduce the systematic positioning error µ q by a factor of 8.

[0065] In addition to the total distance error, other evaluation criteria are considered: the lateral portion of the distance error, the mean angular error, and the ratio of the time in which the assignment between the lanes was correctly identified. Regardless of the source and magnitude of the noise (within the tested areas, as described in Fig.6), the mean lateral error always remains below 0.40 m, the mean angular error below 0.011 rad, and the correct assignment rate above 0.987. These values ​​demonstrate that the identification of the correct lane, the lateral offset, and the angle are highly noise-insensitive. Thus, the distance error introduced by noisy measurements is primarily reflected in a longitudinal offset.

[0066] A real-world test was conducted using a lane-level accurate road map, standard GPS, and a lane detection system. Fig.Figure 7 shows a typical scene of a drive along with the road map and the computed pose. A high-accuracy positioning system with an error of less than 5 cm was used as a reference. For a drive of a total length of 13.2 km in Wolfsburg, Germany, the computed global position had an average total offset of 1.71 m, an average lateral offset of 0.39 m, and an average angular offset of 0.009 rad relative to the reference system. Fig. Figure 7 shows, on the left, lanes of the map combined with a satellite image as a visualization aid, and the global pose measurement 71. The center shows a camera image and lane measurements generated from it. On the right, calculated hidden variables for lane centers 72, 73, and 74 and vehicle poses 75 to 78 are shown, with the pose with the highest probability being pose 76. List of reference symbols 10 vehicles 11 Arrangement determination device 12 Driver assistance system 13 Connection 14 Sensor, Camera 15 Processing device 16 Antenna 17 vehicle bus 18 Wheel sensor 20 procedures 21-28 steps 71 Global Pose Measurement 72-74 lane centers 75-78 vehicle poses

Claims

[1] A method for determining arrangement information for a vehicle, wherein the arrangement information comprises a position of the vehicle (10) and an orientation of the vehicle (10) with respect to a stationary coordinate system, the method comprising: - detecting a local lane arrangement in an environment of the vehicle (10) with sensors (14) of the vehicle (10), wherein the local lane arrangement indicates an arrangement of lanes in the environment of the vehicle (10) with respect to the vehicle (10), - determining a first lane arrangement as a function of the local lane arrangement and a previously determined arrangement information for the vehicle, wherein the first lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, - determining a second lane arrangement as a function of predetermined map material, wherein the second lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, - Determining a similarity information item as a function of a geometric similarity of the first lane arrangement and the second lane arrangement by determining a plurality of pair similarity information items, wherein each of the plurality of pair similarity information items is determined as a function of a geometric similarity of a respective pair of two lanes, wherein a respective pair of two lanes each comprises a lane from the first lane arrangement and a lane from the second lane arrangement, wherein the plurality of pair similarity information items comprise a respective pair similarity information item for each combination of a lane from the first lane arrangement and a lane from the second lane arrangement, and - updating the previously determined arrangement information for the vehicle (10) depending on the similarity information. [2] Method according to claim 1, characterized by that the procedure further comprises: - updating a previously determined lane arrangement hypothesis as a function of the similarity information, wherein the lane arrangement hypothesis specifies a hypothesis for an arrangement of lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system. [3] Method according to claim 1 or 2, characterized by that the arrangement information of the vehicle (10) comprises parameters of a multidimensional normal distribution for the position and orientation of the vehicle (10). [4] Method according to one of the preceding claims, characterized by that a lane of a lane arrangement comprises a point list representing the lane center, each point of the point list having parameters of a multidimensional normal distribution for a position with respect to the stationary coordinate system. [5] Method according to one of the preceding claims, characterized by that the procedure further comprises: - Updating the previously determined arrangement information for the vehicle (10) depending on odometry data of the vehicle (10). [6] Method according to one of the preceding claims, characterized by that the procedure further comprises: - detecting a stationary position of the vehicle (10), wherein the stationary position indicates a position of the vehicle (10) with respect to the stationary coordinate system, and - Updating the previously determined arrangement information for the vehicle (10) depending on the detected stationary position of the vehicle (10). [7] Method according to one of the preceding claims, characterized by that the method is carried out repeatedly, wherein a repetition rate for the repeated execution of the method is in a range of 25 to 400 repetitions per second. [8] Method according to one of the preceding claims, characterized by that determining the similarity information includes: - determining lane information for the first lane arrangement using the vehicle's sensors, the lane information comprising lane marking types and / or stop line positions and / or traffic sign positions, and - Comparing the lane information for the first lane arrangement with map-based lane information for the second lane arrangement. [9] Method according to one of the preceding claims, characterized by that updating the previously determined arrangement information for the vehicle (10) in dependence on the similarity information comprises: - for a respective pair similarity information of the plurality of pair similarity information: determining a respective weighted update information as a function of the respective pair similarity information, wherein the weighted update information comprises arrangement information for the vehicle (10) under the assumption that the lanes of the respective pair relate to the same actual lane, and wherein the weight of the weighted update information indicates a probability that the lanes of the respective pair relate to the same actual lane, and - updating the previously determined arrangement information for the vehicle (10) depending on the weighted update information. [10] Method according to claim 9, characterized bythat the arrangement information of the vehicle (10) and the weighted update information each comprise parameters of a multidimensional normal distribution for the position and orientation of the vehicle (10), wherein updating the previously determined arrangement information for the vehicle (10) as a function of the weighted update information comprises: - Multiplying the normal distribution of the ordering information with the normal distributions of the update information. [11] An arrangement determining device for determining arrangement information for a vehicle, wherein the arrangement information includes a position of the vehicle (10) and an orientation of the vehicle (10) with respect to a stationary coordinate system, the arrangement determining device comprising: - sensors for detecting a local lane arrangement in an environment of the vehicle (10), wherein the local lane arrangement indicates an arrangement of lanes in the environment of the vehicle (10) with respect to the vehicle (10), and - a processing device (15) which is designed to determine a first lane arrangement as a function of the local lane arrangement and a previously determined arrangement information for the vehicle (10), wherein the first lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, to determine a second lane arrangement as a function of predetermined map material, wherein the second lane arrangement indicates an arrangement of at least two lanes in the surroundings of the vehicle (10) with respect to the stationary coordinate system, to determine a similarity information item as a function of a geometric similarity of the first lane arrangement and the second lane arrangement by determining a plurality of pair similarity information items, wherein each of the plurality of pair similarity information items is determined as a function of a geometric similarity of a respective pair of two lanes, wherein a respective pair of two lanes each comprises a lane from the first lane arrangement and a lane from the second lane arrangement, wherein the plurality of pair similarity information items comprise a respective pair similarity information item for each combination of a lane from the first lane arrangement and a lane from the second lane arrangement, and to update the previously determined arrangement information for the vehicle (10) depending on the similarity information. [12] Arrangement determining device according to claim 11, characterized by that the arrangement determining device (11) is designed to carry out the method according to one of claims 1-10. [13] Vehicle comprising: - an arrangement determining device (11) according to claim 11 or 12, and - a driver assistance system (12) which provides an automatic driving function for the vehicle (10) and is coupled to the arrangement determining device (11), wherein the automatic driving function is executed on the basis of the updated arrangement information from the arrangement determining device (11). [14] Vehicle according to claim 13, wherein the arrangement determining device (11) is designed to carry out the method according to claim 2, characterized by that the automatic driving function is executed on the basis of the updated lane arrangement hypothesis by the arrangement determining device (11).

Images