Method and system for capturing the traffic environment of a mobile unit

Abstract

Method for capturing a traffic environment of a mobile unit (1) wherein (3) data records are recorded using a data acquisition unit, with each data record being assigned a time point; For each of the recorded datasets, landmark observations are determined, whereby landmark parameters and object classes are assigned to the landmark observations; The landmark observations are classified as point-based, line-based, or polyline-based objects based on their assigned object classes; Environmental model data are generated based on specific landmark observations, with the environmental model data including a description of the traffic environment; Distance calculations for landmark observations are performed using the environment model data based on their object class and landmark parameters, whereby distance values ​​for the landmark observations are determined, and the landmark observations are assigned to specific landmarks based on these distance values; in the distance calculations - a Mahalanobis distance is determined for point-based objects, - for line-based objects, a modified Mahalanobis distance is determined based on the start and end positions and - for polyline-based objects, an area enclosed by the polyline-based objects is determined, whereby a projection of polylines onto each other is first carried out and the polylines are cut off in such a way that only those sections of the polylines are processed further that are opposite a section of another polyline; Probabilities are assigned to the environmental model data, where the probabilities indicate with what certainty a real landmark was detected, and the assigned probabilities are changed based on landmark observations. the environmental model data are updated based on the probabilities assigned to the landmark observations; and The environmental model data will be output.

Description

[0001] The present invention relates to a method and a system for detecting a traffic environment of a mobile unit, in particular a vehicle.

[0002] For the increasing integration of automated driving functions into modern vehicles, precise vehicle localization is a key prerequisite. It's not enough to know the vehicle's position to within a few meters; even small deviations, such as the width of a lane, can have a significant impact on the journey.

[0003] In order to enable sufficiently accurate and reliable localization of the vehicle, approaches such as landmark-based positioning are being pursued.

[0004] In the method for detecting markings on a road surface described in DE 10 2004 003 850 A1, a laser scanner captures distance images of a detection area in chronological order, and the distance to a detected marking is estimated. Markings are identified by detecting straight lines on the road surface.

[0005] DE 10 2007 020 791 A1 proposes a lane marking detection device in which a center line and lane width are extracted from data from a lidar instrument. Parameter tracking is used.

[0006] Of particular importance in such processes is the management of landmarks detected by the mobile unit's sensors. Several different sensors can be used to ensure reliable detection through redundancy, while complementary sensor principles ensure the most complete detection possible.

[0007] Especially in such cases, there is a need for methods for processing the detected landmarks in order to reliably generate a comprehensive representation of the mobile unit's surroundings. It is the object of the present invention to provide such a method and system.

[0008] The 2016 publication by Steß et al., "Multi-sensor tracking with SPRT in an autonomous vehicle," describes a method for tracking landmarks in an autonomous vehicle. Observations of an environment are recorded using cameras and lidar, and landmarks are detected.

[0009] In the 2015 publication, Steß et al., "Camera-based field of view parameter optimization," a method is described that detects and tracks road markings. The start and end points of the detected markings are used to calculate distances to previously detected markings.

[0010] The method for self-locating a vehicle described in DE 10 2014 221 473 A1 calculates a probability distribution according to which a vehicle is located at a specific position. Location points are detected and tracked in the camera images. This allows an improved determination of an estimated position.

[0011] US 2012 / 0143808 A1 describes a method for detecting and identifying an object, whereby objects can be tracked and artificial intelligence is used to determine a probability for assigning an object to a target object.

[0012] According to the invention, this object is achieved by a method having the features of claim 1 and a system having the features of claim 7. Advantageous embodiments and further developments emerge from the dependent claims.

[0013] In the method according to the invention, data sets with data points are acquired by means of a recording unit, with each data set being assigned a point in time. Landmark observations are determined for each of the acquired data sets, with landmark parameters and object classes being assigned to the landmark observations. Based on the determined landmark observations, environment model data is generated, with the environment model data comprising a description of the traffic environment. Based on the environment model data, distance calculations are carried out for landmark observations based on their object class and their landmark parameters, with distance values ​​being determined for the landmark observations. The landmark observations are assigned to specific landmarks based on the distance values.Probabilities are assigned to the environment model data, where the probabilities indicate the degree of certainty with which a real landmark was detected. The assigned probabilities are modified based on the landmark observations. The environment model data is updated based on the probabilities assigned to the landmark observations, and the environment model data is output.

[0014] This advantageously allows a comprehensive representation of the traffic environment to be captured and made available for further processing. In particular, the method can be applied to localize a mobile unit, such as a vehicle.

[0015] The method according to the invention operates in particular on a vector-based basis. This means that it does not perform a comprehensive spatially resolved mapping of a space, as is the case with point- or grid-based methods. Instead, individual objects are detected, parameterized, and stored based on this parameterization. This allows the method to be operated quickly and with comparatively low memory and computing power requirements, which is particularly advantageous for mobile units, such as those in vehicles.

[0016] In a narrower sense, the term "position" encompasses the coordinates of a point in a coordinate system, whereby a coordinate system can be considered relative to the mobile unit or a global coordinate system. In a broader sense, however, the term "position" should also encompass characteristics that typically fall under the term "pose," in particular an orientation of the mobile unit in a specific direction within the coordinate system. For the sake of simplicity, the term "position" is used below, but the other characteristics of a "pose" are not to be excluded.

[0017] "Landmarks" within the meaning of the invention include, among other things, orientation points and topographical objects that are recognizable and classifiable and suitable for structuring a spatial area or relative to which a position can be defined within the meaning of the invention. Classifiable objects are, in particular, those that can be represented as geometric shapes and parameterized, for example, based on their shape, position, area, length, and width. In particular, landmarks include elements of the traffic infrastructure, such as road markings, traffic signs, lighting systems, bridges, or other traffic structures. Furthermore, they include other structures, such as houses, as well as plants, such as trees, and landscape features, such as elevations.Landmarks should also be recognizable, i.e. their appearance and other properties should remain essentially constant over a certain period of time, in particular in such a way that recognition based on the landmarks is possible over a sufficiently long period of time in order to be able to take into account any changes by means of an update functionality.

[0018] In particular, according to the invention, semantically interpretable elements of the road infrastructure are recorded as landmarks. These can further be grouped according to their geometry and the processing and recording options. In particular, a distinction can be made between point-based landmarks, such as poles, posts, and pillars, which can be detected in particular using lidar; line-based landmarks, such as dashed lines, arrows, and stop lines, which can be detected in particular using lidar or a camera; and polyline-based landmarks, such as solid lines, which can be represented as line sets and for whose detection a camera can be used in particular.

[0019] To characterize a landmark observation, a state vector is generated, whereby the data stored in this state vector differs depending on the type of landmark: The state vector s POa point-based landmark observation includes a position p vrf in the vehicle coordinate system, an uncertainty Σ p and an absolute position p wgs in a global coordinate system. The points are specified as triples p = (x, y, z). Furthermore, a diameter and a height are defined, and a timestamp for the observation and an identification of the observing sensor are stored. Furthermore, fields are provided for a probability value and an identification of a specific landmark, which are used for further processing in the process. For the state vector s DL a line-based landmark observation or the state vector s PL A polyline-based landmark observation requires two or more positions (p1vrf,…pnvrf) and correspondingly many uncertainties (Σ p1 , ..., Σ pn) and absolute positions (p1wgs,…pnwgs) as well as modified parameters such as the width and length of the (poly)line.

[0020] Alternatively or additionally, further parameterizations of a landmark observation, in particular for other types of landmarks, may be provided.

[0021] The "traffic environment" of the mobile unit within the meaning of the invention comprises a spatial environment of a position of the mobile unit, in particular with regard to the properties relevant to traffic in this environment. In particular, the environment of a road user, for example, on a roadway, can be considered according to the invention. The spatial area is essentially determined by the range with which data can be collected in the environment and the environment in which landmarks can be used to localize the mobile unit.

[0022] A "description of the traffic environment" includes, in particular, information about landmarks in the traffic environment. This may include the positions and characteristic properties of the landmarks.

[0023] The acquisition of the data sets in a first step of the method is carried out in a manner known per se. In particular, sensors with various operating principles can be used, for example, lidar, radar, or ultrasonic sensors, visible light or infrared cameras, and time-of-flight cameras. Furthermore, data can be acquired from external devices, in particular from other vehicles or other vehicle-external devices, whereby methods from the field of "car-to-X" communication can be used.

[0024] A captured data set is captured and / or output by a sensor at a specific point in time, such as a single scan from a laser scanner or an image captured by a camera. A data set can comprise the data points of a detection at a specific point in time, or it can accumulate data points from multiple measurements.

[0025] Depending on the type of sensor used, various characteristics of the traffic environment can be detected, particularly with spatially resolved detection. For example, properties of reflection, scattering, transmission, or absorption of electromagnetic radiation, as well as distances, can be recorded. The data points correspond to individual measurements, particularly at different locations within a sensor's detection range.

[0026] In a special case, the data sets can be captured using a laser scanner. A captured data set then comprises, in particular, a single scan of the laser scanner, whereby parameters such as distances and reflection properties at points in the traffic environment can be captured by swiveling a light beam and detecting the reflected light within a specific detection range. The data points correspond to individual measurements, particularly at different angles relative to the laser scanner.

[0027] The laser scanner can capture the data sets periodically. Furthermore, the process can also be executed periodically. This advantageously allows for continuously updated data to be provided, allowing landmarks in the traffic environment to be captured as accurately as possible at any given time. Further processing of the captured landmarks allows for continuous refinement and tracking of landmark-based localization based on the current data.

[0028] In other embodiments, the detection may also be performed in a different manner, such as asynchronously.

[0029] Furthermore, it can be provided that data from additional sensors is collected. In particular, data relating to the movement and guidance of a vehicle can be collected. For example, speed and acceleration can be recorded, as well as the direction of movement and the steering angle.

[0030] Furthermore, it can be provided that data from additional devices is collected, in particular using a data connection. For example, the mobile unit can be connected to other devices via a data connection, and data collected by these devices can be used. The data connection can be established in various known ways, and data can be transmitted in a similar manner, for example using a method for retrieving specific data from a remote device. For example, data collected by other vehicles can be transmitted to a vehicle, or other devices can be used in a similar way.

[0031] In particular, the recorded raw data can be filtered, for example to limit the recorded landmarks to features of the roadway, such as road markings, which are particularly important landmarks for the localization of vehicles.

[0032] The acquired data can be combined into output data for further processing. In particular, in a special case, a single data set is used here. Furthermore, accumulation can be carried out. The suitability of a certain number of data points and / or data sets for further processing, as well as the need for accumulation, also arises from the properties of the acquired data sets, such as the number of data points in a single data set. The output data can be stored in a circular buffer in a conventional manner, with the most recently acquired data being saved while the oldest existing data is deleted. In this way, the data from a specific, recent period can be stored together.

[0033] In a further step, segmentation can now be performed based on the source data. Data points potentially belonging to a landmark are grouped into "segments." For each potential landmark, a subset of the source data is defined for the corresponding data points.

[0034] In a further development, segmentation is performed using a Euclidean Cluster Extraction Algorithm, whereby data points whose distance from each other is less than a predetermined threshold are each assigned to a segment. This advantageously allows for simple and rapid segment determination.

[0035] Alternatively or in addition, other algorithms can be used, such as a Random Sample Consensus Algorithm (RANSAC), whereby various segmentation methods can be implemented.

[0036] The segments are then analyzed using principal component analysis (PCA). Parameters are determined for each segment and stored as "landmark parameters." Landmark parameters are particularly suitable for defining the segment as comprehensively as possible. For example, a geometric shape can be determined that describes the distribution of the data points in a segment.

[0037] To determine the landmark parameters for each segment, the principal component analysis can include determining an approximate ellipse for the segment. The geometric shape of the approximate ellipse advantageously allows for a variety of other geometric shapes to be approximated easily and with high accuracy.

[0038] The parameters of the approximate ellipse allow conclusions to be drawn about the actual dimensions and parameters of the geometric shape. In particular, points and lines can be approximated particularly well using the approximate ellipse. Furthermore, PCA enables a particularly fast and robust analysis of the segments. In particular, the relevant parameters of the approximate ellipse are determined jointly in a single step, so that the number of calculation steps and the computational effort for detecting the landmarks are significantly reduced compared to other known methods. In particular, two- or multi-dimensional data can be easily processed, and a suitable parameterization can be determined. For example, the length and width of an elongated object, such as a line, can be determined particularly quickly and easily.

[0039] In one embodiment, the landmark parameters include at least one relative position relative to the detection unit. This advantageously allows a position determination for the detected object relative to a detection coordinate system, in particular relative to a vehicle coordinate system.

[0040] In particular, this position can be determined together with the other landmark parameters using principal component analysis. Cartesian coordinates can be used, for example, or a position can be determined using distance and angle.

[0041] The origin of the coordinate system can be assumed to be the mobile unit, so that the position is initially determined relative to it. In a subsequent step, a transformation into another coordinate system, in particular a global coordinate system, can then take place.

[0042] In a further development, filtering is performed based on the landmark parameters of the segments. This advantageously selects the segments that are considered to be true landmarks, and further calculations can be restricted to these.

[0043] For filtering, thresholds can be used to verify the plausibility that detected objects correspond to actual landmarks and to avoid detection artifacts. For point-based landmarks, an upper and / or lower limit for the detected radius can be checked. Furthermore, for line-based and polyline-based landmarks, upper and / or lower limits for the detected length and / or width can be used.

[0044] Furthermore, legal regulations can be used that stipulate certain dimensions for certain landmarks of traffic infrastructure, especially road markings. For example, a minimum and / or maximum width and / or length of lines that are accepted as potential road markings can be specified. Furthermore, certain shapes can be recognized and their plausibility for recognition as landmarks can be assessed.

[0045] The information about individual landmarks recorded and determined using a data set is each summarized to form a "landmark observation." This allows the data to be structured. According to the invention, a landmark observation comprises all information about a specific sighting of a specific landmark within a set of initial data. Thus, with repeated application of the method, a series of landmark observations can be determined for the same real landmark if the landmark is sighted multiple times, in particular by multiple sensors and recording units. In particular, the landmark observation can be assigned to the corresponding landmark, for example by means of an identification number, in order to be able to compare multiple landmark observations of the same landmark with one another.

[0046] In a further development, the landmark observations are assigned to elements of a road infrastructure. This allows the data to be advantageously processed in such a way that the landmark observations can be assigned to specific landmarks.

[0047] For example, this method can also be used to determine a match with landmarks captured using additional data sets, enabling further processing. In particular, a position can then be determined, using various landmark-based localization methods.

[0048] In a further step, object classes are determined based on the landmark parameters. This involves classification according to basic geometric shapes, which allow for a basic assignment of the segments to different landmarks.

[0049] According to the invention, the landmark observations are classified as point-based, line-based, or polyline-based objects based on their assigned object classes. This advantageously allows various landmarks, in particular road markings and / or other traffic signs such as posts, to be accurately detected and parameterized.

[0050] Point-, line-, or polyline-based landmarks allow for a particularly useful classification of landmarks, especially road markings. The classification is based on the landmark parameters necessary to fully characterize a landmark, in particular the number of positions to be determined to capture the course of a road marking.

[0051] The classification step can be performed for specific segments. As described above, different parameters can be determined and saved depending on the object class. This enables particularly efficient data management.

[0052] According to one embodiment, the determined landmark parameters for segments of the line-based or polyline-based feature class include a length parameter and a width parameter as well as a start position and an end position. This advantageously provides the most relevant parameters for describing these classes of objects as well as their variances for further analysis.

[0053] Optionally, polyline-based objects can be simplified using the Ramer-Douglas-Peucker algorithm or a similar suitable method.

[0054] In the method according to the invention, the landmark observations are further processed. For this purpose, the data can be transmitted, for example, to a landmark-based localization unit; thus, there is not necessarily a perceptible output for a user, such as in the form of a display.

[0055] In the method according to the invention, environmental model data is determined based on the specific landmark observations, wherein the environmental model data comprises a description of the traffic environment of the mobile unit. The environmental model data thus provides data that is particularly relevant for landmark-based localization of the mobile unit.

[0056] The determined environment model data includes, in particular, data about landmarks and the positions assigned to the landmarks. This allows the environment model data to be determined in such a way that they represent a comprehensive description of the landmarks in the traffic environment of the mobile unit and enable landmark-based localization on their basis.

[0057] In particular, the environment model data includes information about the landmarks in the traffic environment of the mobile unit. For example, object classes assigned to the landmarks can be determined and saved, and the landmark parameters can be included with data about the positions of the landmarks. The requirement that the generated environment model should be consistent particularly applies to those landmarks for which there are multiple landmark observations from multiple sightings: These are merged and tracked over a certain period of time, creating so-called "tracks." The tracks represent observations belonging to a landmark and their tracking based on the positions determined during the observations. The tracks of the environment model data serve to manage the data recorded for a landmark in the traffic environment in such a way that the landmarks are ultimately represented as precisely as possible by the environment model data.

[0058] Since the data sets can be acquired by different sensors, they may be recorded at different times. For example, one sensor may collect data periodically at a specific rate, while another sensor may collect data at a different rate or at a different time interval. Accordingly, different times are assigned to both the data sets and the data of the landmark observations determined from the data sets. In this case, the different times and, in particular, any movement performed in the meantime must be taken into account. For example, a shift in the origin of the coordinate system of the mobile unit between the times of acquisition of different data sets can be taken into account.

[0059] To process landmark observations based on different datasets, possibly from different sensors, tracking is performed in a single step, allowing a landmark to be recognized from different datasets. This allows landmark observations to be confirmed as belonging to a real landmark if they are detected in multiple datasets.

[0060] Therefore, distance calculations for landmark observations are performed using the environment model data based on their object class and landmark parameters. The landmark observations are assigned to specific landmarks based on the distance values. This advantageously allows corresponding landmark observations to be determined very easily and quickly.

[0061] In particular, this step serves to merge newly determined landmark observations with the environment model data. For example, a landmark in the traffic environment was previously observed and is included in the environment model data. In this case, the environment model data already contains a "track" for this landmark. If another landmark observation is added, it must be determined whether the new landmark observation corresponds to an existing track in the environment model data or whether a new landmark has been detected for the first time. The assignment is based on the determined distance values; it is therefore assumed that landmark observations that were recorded essentially at the same location or sufficiently close to one another can be assigned to one another and belong to the same actual landmark.

[0062] The positions of landmarks represented by the tracks of the environment model data are compared based on the landmark parameters of various landmark observations, and distance calculations are used to determine which recorded landmarks belong together. In particular, the distance calculations are performed based on the object classes assigned to the landmarks and landmark observations. This takes into account, in particular, that different landmark shapes require different distance calculation methods. For example, point-based landmarks can be assigned a single position, whereby the geometric distance between such landmarks can be unambiguously determined based on the respective positions.The calculation is complicated for line- and polyline-based landmarks by the fact that they do not have a single, unique position value that can be compared with another position value. Instead, several positions are characteristic of such landmarks, such as start and end positions and intermediate points.

[0063] According to the invention, a Mahalanobis distance is determined for point-based objects during distance calculations, a modified Mahalanobis distance is determined for line-based objects based on the start and end positions, and / or an area enclosed by the polyline-based objects is determined for polyline-based objects. First, a projection of line strings onto each other is performed, and the line strings are truncated so that only those sections of the line strings that are opposite a section of another line string are further processed. This advantageously provides suitable distance metrics for different object classes.

[0064] The method for determining the Mahalanobis distance for point-based landmarks is known per se. This equation is modified for line-based landmarks, each of which has a start and an end position, so that the four possible combinations for the distance measurement are considered together (start position 1 - start position 2; end position 1 - start position 2; start position 1 - end position 2; end position 1 - end position 2). The area enclosed by polyline-based landmarks (lines) is used as a measure of the distance between them. Alternatively or additionally, other suitable methods can be used to determine distance metrics.

[0065] In particular, the data for landmark observations are evaluated for each real landmark, i.e., for each landmark detected in the traffic environment, the data of the corresponding landmark observations are stored and the data of different landmark observations can be processed collectively.

[0066] To further determine whether a landmark observation corresponds to a real landmark, probabilities can be assigned to the registered landmarks of the environment model, indicating the degree of certainty with which a real landmark was detected. The probability value can be changed by comparing the data of different landmark observations assigned to the same landmark.

[0067] In a further development, the probabilities assigned to the environment model data are modified using a sequential probability ratio test. These probabilities are used to check whether a landmark observation corresponds to an actual detection of a landmark in the traffic environment. The probabilities assigned to the landmarks of the environment model, i.e., the tracks, are thus calculated using a sequential probability ratio test. This advantageously allows for the application of a statistical test that allows for a particularly reliable determination of probabilities in a variable data set.

[0068] In one design, the probability of a landmark observation is increased if another landmark observation is associated with it, and the probability of a landmark observation is decreased if it is not confirmed by another landmark observation. This advantageously allows the probability to be determined with increasing accuracy in an iterative process.

[0069] In a further refinement, the landmark observations of the environment model are accepted or rejected based on a comparison of their assigned probabilities with a threshold value. This advantageously allows for an iterative testing process to determine whether a decision for or against accepting a detected landmark as actually present can be assumed. This allows for particularly simple and reliable identification and deletion of false detections.

[0070] In a further development, the environment model data is assigned to landmarks and includes the positions of the landmarks. The environment model data with the confirmed landmarks of the tracks can thus advantageously be further used to represent a description of the traffic environment.

[0071] The environment model data can, for example, correspond to a map of the mobile unit's traffic environment. In particular, a consistent environment model is generated, meaning that the environment model data is generated based on the data collected from multiple data sets in such a way that the collected data sets are synchronized and spatially consistent relative to each other.

[0072] The environment model data is then output according to the method according to the invention, whereby further processing of the data for landmark-based localization can be carried out. For this purpose, the environment model data can, for example, be transferred to another unit for evaluation or can be provided to a program module for processing.

[0073] With landmark-based localization of the mobile unit using the environment model data determined according to the invention, an estimated position of the mobile unit can be detected, and reference map data can be retrieved based on the detected estimated position, wherein the reference map data includes reference positions of landmarks. Based on the environment model data and the reference map data, the position of the mobile unit can then be determined using a particle filter-based method. Furthermore, alternatively or additionally, the position can be determined using other methods, such as a Kalman filter. This advantageously allows localization based on reference map data.

[0074] The reference map data is retrieved based on an estimated position of the mobile unit. The estimated position is recorded using conventional methods, for example, using a satellite positioning system such as the Global Positioning System (GPS). The retrieval itself also occurs in a conventional manner, in particular either from a storage device of the mobile unit or via a data connection to an external unit.

[0075] The estimated position can be acquired and the reference map data retrieved periodically. This advantageously ensures that suitable reference map data is always available and can be used for localization. For example, the retrieval can occur every 10 seconds or at another specified time interval. Furthermore, the retrieval can be performed regularly based on other criteria, such as a distance from the last retrieval position, so that the reference map data is retrieved whenever the mobile unit's position has changed by a certain amount.

[0076] In the particle filter-based method, a multitude of statistically distributed hypotheses about the actual position of the mobile unit are generated, and these hypotheses can be represented as individual particles. The particles are assigned probabilities that they correspond to the true position of the mobile unit, and based on these probabilities, the hypothesis that can be assumed with the highest probability to be the true position of the mobile unit is determined. This position can then be output, for example, by transmitting it to another unit such as a driver assistance system or a navigation system.

[0077] The system according to the invention of the type mentioned above comprises a recording unit by which data sets with data points can be recorded, with each data set being assignable to a point in time. It further comprises a landmark recognition unit by which landmark observations can be determined for the recorded data sets, with landmark parameters and object classes being assignable to the landmark observations, and with the landmark observations being classifiable as point-based, line-based, or polyline-based objects based on the object classes assigned to them. It further comprises a computing unit by which environment model data can be generated based on the determined landmark observations, with the environment model data comprising a description of the traffic environment, and a tracking unit by which distance calculations for landmark observations can be performed based on their object class and their landmark parameters.Where distance values ​​for the landmark observations can be determined, whereby in the distance calculations for point-based objects, a Mahalanobis distance can be determined, for line-based objects, a modified Mahalanobis distance can be determined based on the start and end positions, and for polyline-based objects, an area enclosed by the polyline-based objects can be determined. First, a projection of line strings onto each other is carried out and the line strings are truncated such that only those sections of the line strings are further processed that are opposite a section of another line string. The computing unit can assign probabilities to the environment model data, whereby the probabilities indicate the certainty with which a real landmark was detected.and the associated probabilities are variable based on the landmark observations; furthermore, the environment model data is updateable based on the probabilities associated with the landmark observations. Finally, the environment model data is outputtable.

[0078] The system according to the invention is particularly designed to implement the above-described method according to the invention. The device thus has the same advantages as the method according to the invention.

[0079] Furthermore, it can be provided that map data can be generated and output based on the environment model data. The map data includes information about the landmarks, in particular their positions. It can also be provided that the map data can be transferred to an external server and / or other devices.

[0080] In one embodiment, the detection unit comprises at least two different sensor types. This advantageously allows data to be acquired in a complementary manner, and multiple detections create redundancy, making the method more robust. Different sensor types, in particular, utilize different detection principles.

[0081] For example, a camera, a laser scanner, a radar detector and / or another sensor may be provided.

[0082] Furthermore, the system can include additional sensors. In particular, data relating to the movement of the mobile unit, in particular a vehicle, can be recorded. For example, the speed and acceleration of a vehicle can be recorded, as well as the direction of movement and the steering angle.

[0083] In a further development of the system, the environment model data can be transmitted to a positioning unit, in particular to a unit for landmark-based localization. This advantageously allows the acquired and determined data to be used for localization. In particular, landmark-based localization of a vehicle can be performed.

[0084] Alternatively or additionally, the environment model data can also be transmitted to other devices, such as driver assistance systems that can support or independently execute vehicle control functions using the detected landmarks, or remote devices, such as other vehicles or devices external to the vehicle.

[0085] The invention will now be explained using embodiments with reference to the drawings. Fig. 1A and Fig. 1B show an embodiment of the system according to the invention, Fig. 2 shows an embodiment of a landmark-based localization, Fig. 3 shows an embodiment of a detection of landmarks, Fig. 4A and Fig. 4B show the generation of a consistent environment model, Fig. 5 shows performing a particle filter-based localization and Fig. 6 shows an embodiment of an update functionality.

[0086] With reference to the Fig. 1A and Fig. 1B describes an embodiment of the system according to the invention.

[0087] In the exemplary embodiment, a vehicle 1 is provided as the mobile unit 1.

[0088] The vehicle 1 comprises a detection unit 3, which in turn comprises a laser scanner 3a, a camera 3b, and a GPS sensor 3c. The detection unit 3 is coupled to a control unit 4. Also coupled to the control unit 3 are a computing unit 6, a driver assistance system 12, and an interface 5. The computing unit 6 comprises an accumulation unit 7, a segmentation unit 8, a landmark recognition unit 9, and a tracking unit 10. The control unit further comprises an output unit 11.

[0089] In the exemplary embodiment, interface 5 is connected to an external server 2 via data communication. The data connection is established via interface 5 in a conventional manner and, in the exemplary embodiment, is wirelessly established via the Internet. The connection is established via a telephone network. In further exemplary embodiments, the connection is established in a different way, for example, indirectly via a mobile phone connected to the Internet, which can itself establish a connection to external server 2.

[0090] In further embodiments, the detection unit 3 may alternatively or additionally comprise further sensors, in particular using other detection principles. For example, lidar, radar, or ultrasonic sensors, cameras operating in the visible light range or infrared cameras, as well as time-of-flight or stereo cameras may be provided.

[0091] The GPS sensor 3c provided in the exemplary embodiment is formed in a conventional manner and enables the detection of a position using a navigation satellite system such as the Global Positioning System (GPS). Alternatively or additionally, other positioning systems can be used to determine the position of the vehicle 1.

[0092] Furthermore, the exemplary embodiment provides for data to be acquired from additional sensors of the vehicle 1. Data relating to the movement and guidance of the vehicle 1 is acquired. In particular, the speed and acceleration are recorded, as well as the direction of movement and the steering angle.

[0093] The driver assistance system 12 is a device of the vehicle 1 that supports the driver in driving the vehicle 1. The driver assistance system 12 in the exemplary embodiment can act purely as an information system that supports the driver, but it can also fully or partially automatically influence the movement of the vehicle 1 by intervening in control functions.

[0094] Without the driver assistance system 12, the driver directly influences the movement of the vehicle 1. At most, signals or movements from control elements operated by the driver, such as the pedals, the gearshift lever, or the steering wheel, are transmitted to corresponding devices of the vehicle 1, which influence the movement of the vehicle 1. Such movement of the vehicle 1 corresponds to the lowest degree of automation. With a higher degree of automation, some of the devices that serve the movement of the vehicle 1 are automatically intervened in. For example, the steering of the vehicle 1 or the acceleration in a positive or negative direction is intervened in. With an even higher degree of automation, devices of the vehicle 1 are intervened in to such an extent that certain types of movement of the vehicle 1, for example driving straight ahead, can be carried out automatically.At the highest level of automation, a given route can be driven essentially automatically.

[0095] To operate the driver assistance system 12 provided in the exemplary embodiment, the most precise knowledge possible of the position of the vehicle 1 is required. In particular, knowledge of an exact position on the roadway is required, for example, to allow guidance of the vehicle 1 in the transverse and / or longitudinal directions and, for example, to be able to automatically navigate a specific lane or trajectory.

[0096] In Fig. Figure 1B shows some elements of a typical traffic environment for vehicle 1. The drawings are, of course, to be understood as schematic sketches, in which the scales and proportions used are irrelevant.

[0097] The traffic environment is understood to be the spatial surroundings of the position of vehicle 1, especially the properties relevant for traffic in this environment. The traffic environment can therefore include, for example, other road users in the vicinity of vehicle 1 (in Fig. 1B, these have not been shown for simplicity. Furthermore, it can include road markings and traffic signs, infrastructure facilities, and other features, such as buildings, vegetation, or topographical features in the surrounding area. The spatial area is essentially determined by the range with which data can be collected in the surrounding area and the environment in which landmarks can be used for localization.

[0098] Vehicle 1 is located in a lane 18a of a roadway 18, which further comprises a further lane 18b. A guide line 16, comprising several sub-lines, runs between lanes 18a and 18b and is adjoined by a lane marking 15 in the form of a solid line. The roadway 18 is further delimited at its outer edges by lane markings 14. Delineators 13 are also arranged at the edge of the roadway 18.

[0099] Furthermore, a detection range 17 of the detection unit 3 of the vehicle 1 is indicated. In the exemplary embodiment, the laser scanner 3a and the camera 3b of the detection unit 3 of the vehicle 1 are configured to detect data about the traffic environment of the vehicle 1 in the detection range 17. Different sensors of the detection unit 3 can have different detection ranges.

[0100] With reference to Fig. 2, an embodiment of a landmark-based localization is explained as a method for determining a position of a vehicle 1. In this case, the Fig. 1A and Fig. 1B explained system.

[0101] In a first step S1a to S1n, data records are recorded by a number of n sensors of the recording unit 3 of the vehicle 1, for example by the laser scanner 3a and the camera 3b. Fig. 2 a separate step S1a to S1n is provided for a number of n sensors.

[0102] In further embodiments, data records can alternatively or additionally be acquired by sensors of other vehicles and / or devices. Data can then be transmitted to vehicle 1 via a data connection, in particular directly from the vehicles and / or devices that acquired the data, or indirectly via server 2. In this case, processing can take place outside of vehicle 1, for example, by server 2 or the other vehicles and / or devices, or the data records can be transmitted to vehicle 1 and further processed there.

[0103] The acquired data are evaluated in a further step S2a to S2n, where landmarks are detected and landmark observations are generated. Fig. 2 a separate step S2a to S2n is provided for all n sensors.

[0104] The "landmark observations" include all information about a specific sighting of a specific landmark within the data collected in steps S1a to S1n. Each landmark observation is assigned a time point, in particular the time point at which the corresponding data sets were collected. Furthermore, the landmark observations include relative positions relative to vehicle 1 as well as parameters that characterize the state of the landmark detected during a sighting, such as the extent of the detected landmark.

[0105] The part of the method carried out in steps S1a to S1n and S2a to S2n is described below with reference to Fig. 3 is explained in more detail.

[0106] In a further step S3, tracking and fusion of the previously acquired data is performed. This generates data from a consistent environment model, i.e., the information about the traffic environment of vehicle 1 is combined in such a way that the data relevant for landmark-based localization is available. In particular, the environment model includes data from landmark observations, such as data about landmarks in the traffic environment and the positions of the landmarks.

[0107] The generation of a consistent environment model carried out in step S3 is described below with reference to Fig. 4 is explained in more detail.

[0108] In a step S4, an estimated position of the vehicle 1 is detected. In the exemplary embodiment, this is carried out using the GPS sensor 3c of the detection unit 3; in further exemplary embodiments, other methods for determining the estimated position can be used alternatively or additionally. It is assumed that the accuracy with which the estimated position of the vehicle 1 can be determined is insufficient for the operation of the driver assistance system 12. For example, the estimated position can be determined with an accuracy of up to 1 m, whereas guiding the vehicle 1 along a trajectory requires significantly higher accuracy.

[0109] In the exemplary embodiment, the estimated position is transmitted in a step S6 by means of the interface 5 to the external server 2, which provides reference map data in the sense of an “electronic horizon” and transmits it to the vehicle 1. Step S6 takes place in particular as part of a retrieval of the reference map data from the external server 3. The reference map data comprise data on landmarks in the traffic environment of the vehicle 1. With the aid of the estimated position, it can be determined which data relevant to the traffic environment of the vehicle 1 the reference map data should comprise.

[0110] In particular, the reference map data and the data of the environment model are structured in such a way that they can be compared with each other, for example by comparing the positions of landmarks in the respective data.

[0111] In further embodiments, it is provided that the reference map data is retrieved alternatively or additionally from a storage unit of the vehicle 1. In this case, a data connection to the external server 2 is not required, and the method can be operated, for example, under conditions of an incomplete network connection.

[0112] In a further step S5, an ego-motion estimation is performed. In particular, data from vehicle sensors is used, such as data for speed, acceleration, inclination, direction, steering angle, and other data relevant to the movement of vehicle 1. Based on this data, it is possible to determine how vehicle 1 is moving, particularly with greater accuracy determining the position of vehicle 1 at the time a data set is acquired by acquisition unit 3. In particular, this allows the change in the position of vehicle 1 since the last acquisition of an estimated position to be determined.

[0113] The estimated position and self-motion estimation determined in steps S4 and S5 are taken into account alongside the data from the landmark observations for the creation of the consistent environment image by tracking and fusion in step S3.

[0114] Based on the environment image and the reference map data, a particle filter-based localization is performed in step S7. Hypothesis for the position is statistically determined based on the data from the environment model and the reference map data, with the hypotheses represented as "particles." Using a weighting procedure based on the reference map data, the probabilities of the hypotheses are determined, and the most probable position is determined. This is the position of vehicle 1 determined by the procedure.

[0115] Further details of the method used in step S7 are given below with reference to Fig. 5 is explained in more detail.

[0116] In a further step S8, the position is output, wherein in the exemplary embodiment, the determined position is transmitted to the driver assistance system 12 via the output unit 11. Alternatively or additionally, the position can be transmitted in this step to various units of the vehicle or to units external to the vehicle.

[0117] With reference to Fig. 3, an embodiment of landmark detection is explained. This is based on the Fig. 1A and Fig. 1B. In particular, the steps described below can be considered as part of the system described above with reference to Fig. This can be understood as the landmark-based localization explained in Section 2. The method explained here for a sensor can be applied in particular to the data collected by the system's sensors.

[0118] In a first step S1a, data sets are acquired by one of the sensors of the detection unit 3 of the vehicle 1. In particular, the special case of data being acquired by the laser scanner 3a is considered here; however, the method can be transferred to other sensors with appropriate modifications.

[0119] The acquired data sets comprise data points, with each individual data set comprising the data points acquired during a period of scanning the laser beam of the laser scanner 3a. In particular, a distance value is recorded for each data point, as well as data on the reflection of the surface from which the laser beam is reflected. The data is acquired periodically by regularly scanning the laser beam over a specific angular range.

[0120] Optionally, filtering can be performed in this first step S1a. In particular, a suitable laser scanner 3a can already provide an integrated classification of the data points, which includes, for example, information about whether a surface point on the roadway 18 was detected for a data point. For example, all other data points except those on the road can be neglected if road markings are to be recorded as landmarks.

[0121] In a further step S21a, an accumulation is performed. In a special case, the data points of a single data set, in this case a scan of the laser scanner 3a, may already form a sufficient database, so that an accumulation of the data points of one data set is performed. In other cases, the data points of several acquired data sets can be accumulated. The accumulated data form the initial data, which is then used for further processing.

[0122] In particular, the data points can be stored in a ring buffer, where the newly acquired data points are always saved while the older data points are simultaneously deleted. This means that the most recent data for a specific period or data volume is always provided.

[0123] In a further step S22a, segmentation is performed based on the initial data. Data points that appear to belong together are assigned to a segment. In particular, a Euclidean Cluster Extraction Algorithm can be used for this purpose, in which points that are less than a certain maximum distance apart are assigned to a common segment. Alternatively or additionally, other segmentation methods can be used.

[0124] In a subsequent step S23a, the segments are analyzed using principal component analysis (PCA), and an approximate ellipse is determined for each segment. Landmark parameters are determined for each segment, which, in particular, uniquely define the determined approximate ellipse. In particular, the landmark parameters include a direction parameter, a length parameter, a width parameter, and a start and end position. Furthermore, the variances for the respective values ​​can be determined. The landmark parameters can be determined using PCA, in particular, in one step.

[0125] In a step S24a, the segments are classified based on the determined landmark parameters. Three classes are provided, namely point-based, line-based, and polyline-based landmarks. Point-based landmarks are characterized by a specific extent and a position. Line-based landmarks have a start and end position, while polyline-based landmarks have additional intermediate positions.

[0126] For example, continuous lines can be recognized as polyline-based landmarks because neither a clear start nor a clear end point is detected for them. In particular, such lines can be characterized as continuous lines that extend beyond the detection area 17 of the detection unit 2 and are not completely covered by the data points.

[0127] In particular, positions are defined for the landmark parameters. This means that for an approximate ellipse determined using PCA, at least one position is determined, which is initially specified relative to the detection unit 3 or the vehicle 1.

[0128] Based on the determined landmark parameters, a filtering process can be performed, which particularly includes a plausibility check. This check checks whether the length and width of detected line-based landmarks comply with legal regulations regarding the dimensions of road markings formed in this way. In particular, the landmark observations are each assigned to elements of a road infrastructure, such as parts or entire elements of the Fig. 1B, the roadway marking 14, the lane marking or the guide post 13.

[0129] Furthermore, based on certain variances, it can be determined whether a landmark, such as a lane marking, has been detected with sufficient certainty. Additional filtering steps can be applied to exclude poorly or incorrectly detected landmarks.

[0130] Finally, in step S25a, landmark observations are generated. This means that the information determined for the acquired data points is summarized in such a way that they include all data relevant for further processing for landmark-based localization. The individual landmark observations are each assigned to a landmark. This means that a landmark observation comprises data determined during a sighting of the landmark and can therefore be assigned to the detected state of a landmark at the time of acquisition.

[0131] The generated landmark observations specify in particular a state vector of a landmark, whereby the data for this differs depending on the type of landmark: The state vector s PO a point-based landmark observation is given as sPO=(pvrf∑ppwgsDiameterHeightTimestampSensorIDBeliefID), the state vector s DL a line-based landmark observation as sDL=((p1vrf,p2vrf)(∑p1,∑p2)(p1wgs,p2wgs)LengthWidthTimestampSensorIDBeliefID) and the state vector s PL a polyline-based landmark observation as sPL=((p1vrf,…,pnvrf)(∑p1,…,∑pn)(p1wgs,…,pnwgs)LongitudeTimestampSensorIDBeliefID).

[0132] In this case, pnvrf Coordinates in the vehicle coordinate system and pnwgs Coordinates in the global coordinate system, where each point is specified as a triple p = (x, y, z). Furthermore, uncertainties Σ pn The time of detection is indicated in the “Timestamp” field, as is the identification of the sensor as “Sensor ID and the identification of the landmark observation as "ID." The "Belief" field is required for statistical entries of a probability during further processing of the landmark observations. The characterization of the recorded landmarks is specified by the "Diameter," "Height," "Length," and "Width" fields.

[0133] Alternatively or additionally, the state vectors may include other elements, such as an area or a reflectance.

[0134] The landmark observations generated in step S25a can be output and transferred to a module or unit for further processing. In particular, the landmark observations related to Fig. Step S3 described in Figure 2 can be performed. Furthermore, landmark observations from the n sensors can be summarized, for example, in a specific data structure, using, in particular, the state vectors described above.

[0135] With reference to the Fig. 4A and Fig. 4B illustrates the generation of a consistent environment model. This model is based on the Fig. 1A and Fig. 1B. In particular, the steps described below can be considered as part of the system described above with reference to Fig. This can be understood as the landmark-based localization explained in Section 2. In landmark-based localization, a first layer involves sensor- and landmark-specific preprocessing of the raw data, and a second layer is used to build a consistent environment model using a tracking and fusion process. Finally, a third layer creates a map based on the environment model.

[0136] The described method steps can be combined, in particular, in a tracking and fusion module, which is carried out in particular by the tracking unit 10. The recorded landmark observations are fused and stabilized into a consistent landmark environment model, hereinafter referred to as the "environment model." The problems to be solved relate in particular to the fact that the data sets are recorded at different times and therefore at different positions during a movement of the vehicle 1. In addition, false detections must be identified and removed.

[0137] First, landmark observations are received, each of which is assigned to specific times and relative positions relative to vehicle 1. This is done in particular using the method described above with reference to Fig. 3. This preparatory step describes a sensor-based layer of the process. Based on the estimated position (see Fig. 2, step S4) and the egomotion estimation (see Fig. 2, step S5), estimated absolute positions are assigned to the landmark observations. Thus, a coordinate system can be tracked during the continuous movement of vehicle 1, and the relative coordinate system of vehicle 1 can be transformed into a global coordinate system.

[0138] In a first step S31, a time synchronization is carried out. This is Fig. 4B is shown in detail. Parallel to a time axis (t), axes are shown for each sensor 3a, 3b, 3n of the acquisition unit 3. Furthermore, an axis (T) is shown for the so-called tracks, into which the landmark observations are merged at specific times. A track represents the data acquired, merged, and synchronized to a real landmark.

[0139] At certain times and t , which are shown as dashed lines, data about the self-motion of vehicle 1 are received. In each case, sets of landmark observations Btn provided, which are output in particular as a data object by one of the n sensors. The landmark observations Btn are shown here as bold vertical lines. The landmark observations Btn are not recorded synchronously with each other and with the vehicle's own motion data, so the movement must be compensated for during the time difference between two points in time in order to compare the data. This is indicated by horizontal dashed arrows. The landmark observations Btn are buffered until data for a new movement step is available. Then the landmark observations Btn for each sensor 3a, 3b, 3n for the times u t synchronized. Curved arrows pointing downwards indicate that this synchronized data belongs to tracks T t The tracks created in this way are tracked with each movement step. This means that the landmark observations of the Bt−1n and T t-1 in the coordinate system at time t of the motion state u t before.

[0140] In a further step S32, distance calculations are made for landmark observations and landmarks of the tracks in order to find corresponding landmark observations of the Btn the tracks T t First, point-, line-, and polyline-based landmarks can be grouped together, since only they can correspond with each other.

[0141] Gating is carried out using known methods, whereby only landmark observations Btn in a specific environment of the respective tracks T t be eligible for association.

[0142] It is now determined whether a landmark observation Btn a specific track T t This is determined based on the spatial proximity between the two points. To calculate the distances, the following methods are used in the example:

[0143] For point-based landmarks, the state vector s PO a point-based landmark, where the position at time t is given as point p = (x, y, z) in the vehicle coordinate system vrf. Now, a distance function d PO between a point ph→ a landmark observation bhsPO and one point pr→ a track trsPO be calculated. A vector between ph→ and pr→ given as z→pr,ph. A calculation of the Mahalanobis distance is carried out using the equation dPO(bhsPO,trsPO)=(z→pr,ph)T⋅S−1⋅z→pr,ph, where S -1 the inverse covariance matrix of the track landmark trsPO designated.

[0144] For line-based landmarks, the landmark observations include start and end points. Determining a geometric distance is not so unambiguous using typical methods, as significantly different results can be expected if, for example, the start and end positions are swapped. For this reason, the distance metric is modified here according to the Malahanobis equation.

[0145] Considered are a landmark observation bisDL with starting point sl→ and endpoint el→ and a landmark of a track tosDL with starting point like this→ and endpoint eo→. Distances between start and end points of bisDL and tosDL can be calculated, resulting in four possible combinations (distance measurements start position 1 - start position 2; end position 1 - start position 2; start position 1 - end position 2; end position 1 - end position 2): κs0,si=(so→−sl→)T⋅S−1⋅(so→−sl→), κe0,ei=(eo→−el→)T⋅S−1⋅(eo→−e→), κs0,ei=(so→−el→)T⋅S−1⋅(so→−el→), κe0,si=(eo→−sl→)T⋅S−1⋅(eo→−sl→), where S -1 again the inverse covariance matrix of the track landmark tosDL is called.

[0146] Subsequently, values ​​for a root mean square (RMS) can be calculated according to the formula: QMss,ee=(κso,si)2+(κeo,ei)22 QMse,es=(κso,ei)2+(κeo,si)22

[0147] Finally, the distance function d DL calculated according to: dDL(bisDL,tosDL)=(DQVSss,ee,DQVSse,es).

[0148] Alternatively or additionally, other methods can be used to determine the distance metric.

[0149] Another distance metric is defined for polyline-based landmarks, i.e., line strings. In addition to the start and end positions, intermediate points are also considered to determine a distance measure. In particular, an area between the line strings is determined.

[0150] There will be a landmark observation bksPL considered, whose course is a connected list of points (p1, p2, ..., p i ) is modeled, and a landmark observation tlsPL, with the connected point list (p1, p2, ..., p q). In order to determine the area enclosed between the polylines, the line strings are first projected onto each other and the line strings are clipped in such a way that only those sections of the line strings are processed which are opposite a section of another line string.

[0151] To do this, the points along the two lines can be determined where a perpendicular connection to an endpoint of the other line can be determined. The truncation is performed at these points. This prevents, for example, differences in the length of the lines from leading to an overestimation or underestimation of the distance.

[0152] The area enclosed by the lines is calculated, for example, using the Gaussian trapezoidal formula. To do this, the polygon enclosed by the lines is first divided into subpolygons based on intersections of the lines, if any. The areas of the subpolygons are then summed and normalized based on the length of the line segments.

[0153] The closest distance is used as the criterion for assigning the corresponding landmark observations to the landmarks of the tracks, using the greedy nearest neighbor method. Alternatively or additionally, other methods, such as the Hungarian method, can be used for assignment.

[0154] In a further step S33, track management is carried out, whereby probabilities are assigned to the landmarks and changed, whereby a Sequential Probability Ratio Test (SPRT) is applied to check whether a detected landmark corresponds to an actual landmark in the traffic environment of the vehicle 1.

[0155] For example, if a landmark is observed at one time and no longer at another time, it can be checked whether a real landmark was detected or whether a false detection occurred. On the other hand, if a landmark is detected in several consecutive landmark observations, this is considered confirmation, and the value of the assigned probability increases.

[0156] In the exemplary embodiment, the detected landmarks are initially assigned the same probability. The probability is increased if the landmark is confirmed and decreased if the landmark is not confirmed. If the probability value exceeds or falls below a certain threshold, the hypothesis that the detection is an actual landmark is accepted or rejected, respectively. This determines which landmarks should continue to be considered using the SPRT as a statistical test.

[0157] This involves testing a null hypothesis H0 ("All observations originate from a false detection") and an alternative hypothesis H1 ("All observations originate from a true detection"). This tests the degree of certainty that a real landmark was detected based on a landmark observation. This test is performed using the likelihood ratio LR: LR=p(D|H1)p(D|H0)=p(D|H1)1−p(D|H1)=PTPF=PT1−PT, where P T and P F denotes the probabilities corresponding to the hypotheses (H0 / H1 or true / false) and D denotes the data.

[0158] To explain the procedure, individual landmarks observed at time t are referred to as z t Furthermore, it is assumed that the procedure carried out up to this point has already identified all the landmarks of the measurements B t , for which the distance functions do not provide a corresponding landmark in the tracks T t was found when potentially new tracks in T t The set of observations at time t is Z t The observation of the i-th landmark z at time t is called zti∈Zt designated.

[0159] The observations are assigned an initialization probability of L0 at the time of data input. The same value can be chosen for the initialization for all newly observed landmarks, or a sensor-specific value can be assigned. To avoid numerical instabilities, the log-likelihood quotient LLR can be i for the i-th landmark. The probability that the i-th landmark originates from a true detection (alternative hypothesis H1) is then developed across the detection times based on the corresponding observed landmarks: LLRi=ln(L0)+ln(∑t=1tp(zti|H1)1−p(zti|H1))

[0160] This value is used to check whether LLR i exceeds (A) or falls below (B) a certain threshold: B=ln(β1−α)≤LLRi≤ln(1−βα)=A

[0161] LLR icorresponds to a probability for the respective i-th landmark of the environment model; this probability is understood as the confidence value for the track assigned to the respective landmark. In particular, the "Belief" field of the state vector explained above can be used for this purpose. The system design parameters α and β are specified and describe the probability of confirming a false track or deleting a true track. Furthermore, the probability value of a landmark is increased or decreased depending on whether it is confirmed or not by further landmark observations. If this value is exceeded or not met, the iterative process is aborted, and the landmark is confirmed or rejected: {LLRi(t)≥AConfirm TrackB≤LLRi(t)≤AContinue TestLLRi(t)≤BTrack delete

[0162] Finally, in step S34, environment model data is generated, in particular, in a "Track Innovation" step. The set of tracked landmarks in the environment model is updated. This includes data about the landmarks detected in the traffic environment of vehicle 1, in particular their positions. In particular, the positions, uncertainties, and confidence values ​​of the landmark observations are updated based on the data determined in the previous method steps.

[0163] Based on the confirmed tracks, the associated landmarks can be included in a map. This map includes the determined and optimized positions for the landmarks.

[0164] Various known methods can be used to update positions. For example, if several specific positions of a landmark exist, the most recently observed position can be adopted, exponential smoothing can be performed, a moving or arithmetic average can be calculated, or—if the landmark has a length—a length-based weighting can be used to average the position.

[0165] The environment model data is structured so that it can be compared with the reference map data. In particular, the positions of specific landmarks can be compared.

[0166] The environment model data is then output, i.e. transferred to another module or unit for further processing.

[0167] With reference to Fig. 5 explains particle filter-based localization. This is based on the Fig. 1A and Fig. 1B. In particular, the steps described below can be considered as part of the system described above with reference to Fig. 2 explained landmark-based localization.

[0168] In the particle-based method, a particle cloud is generated, with each particle representing a position hypothesis, i.e., a hypothetical position of vehicle 1. The "particle cloud" thus refers to a set of particles. Each particle thus represents a particle state, which is understood as a hypothesis for the state of vehicle 1. A statistical test is then performed to determine which hypothesis, i.e., which particle, best matches the landmark observations with the reference map data. An exemplary embodiment of the method is described in detail below.

[0169] In a first step S40, a check is made to determine whether initialization has already been performed. If this is not the case, initialization is performed in a further step S41. During initialization, as described above for step S4, the estimated position of vehicle 1 is recorded at a specific time, for example, using the GPS sensor 3c, whereby a relatively high degree of inaccuracy is assumed. Furthermore, data on the vehicle's own motion can be taken into account.

[0170] By applying a motion model in step S42, a tracking of the position based on the vehicle's own motion is determined, i.e., the change in the position of vehicle 1 since the last acquisition of the estimated position is taken into account. For initialization, a stored estimated position, such as the last trip of vehicle 1, or a position entered by the user can also be used.

[0171] A corresponding transformation of the particle cloud is performed to compensate for the shift in the position of vehicle 1 due to its own motion. Here, too, an inaccuracy of the corresponding sensors and thus also of the resulting results must be assumed. In particular, uncertainties in sensors for detecting the vehicle's own position (e.g., via GPS) or its own motion (e.g., via odometry) can be taken into account.

[0172] In a further step S43, a sensor model is applied. Sensor-induced measurement noise is taken into account and the probability with which the landmark observations were recorded, assuming the respective hypothetical particle state, is determined.

[0173] This probability results in a weighting for the individual hypotheses, which is assigned to the hypotheses in step S44. In particular, a normalization of the probabilities can be performed.

[0174] In a further step S45, smoothing is performed to stabilize the results, smoothing out significant fluctuations in the probabilities for individual particles. Such fluctuations can occur particularly when significant differences occur during the detection of a landmark at successive points in time, for example, due to measurement errors, obscured landmarks, or the disappearance of the landmarks from the detection area 17 of the detection unit 3.

[0175] For example, the smoothed weighting ω f,t of the particle f in time step t can be calculated according to: ωf,t=αωf,t+(1−α)ωf,t−1

[0176] A damping factor α is chosen empirically and the strength of the damping can be further determined by a suitable choice of the time window between time steps t and (t - 1).

[0177] In step S46, the position corrected according to the preceding steps is output. In particular, the state of the particle with the maximum assigned probability is taken into account.

[0178] In particular, during output, the determined position is transferred to a module or unit for further processing; in particular, the output can be made to the driver assistance system 12.

[0179] Furthermore, resampling can be performed in a further step S47. This may be necessary if, through iteration of the method described above, the number of particles converges to a single particle. This is particularly due to the fact that the scatter of the particle cloud increases in each step due to accumulated measurement errors while the vehicle's own motion is repeatedly tracked. As more and more particles move further and further away from the most probable position, they are assigned a correspondingly low probability. This leads to fewer and fewer particles with a particularly high probability being considered.After an initial scatter of particles has been generated during sampling, so-called “recovery particles” are generated during further iterations of the procedure by resampling, which add new hypotheses to the particle cloud in order to make the statistical selection of the most probable hypothesis based on a sufficiently large database.

[0180] With reference to Fig. 6 an example of an update functionality is explained. This is based on the Fig. 1A and Fig. 1B and the system explained with reference to Fig. 2 explained procedures.

[0181] In step S50, the estimated position of vehicle 1 is first detected. This is done, for example, by the GPS sensor 3c, as described above. The estimated position is transmitted to the external server 2 when reference map data is retrieved.

[0182] Based on the estimated position, reference map data is generated by the external server 3 in a step S51 and transmitted to the vehicle 1. The server 2 accesses a reference map database and generates the reference map data in such a way that it includes all information relevant for the landmark-based localization of the vehicle 1, in particular landmarks and their positions in the traffic environment of the vehicle 1.

[0183] Vehicle 1 now performs landmark-based localization using the reference map data. This allows differences between the landmark observations recorded by Vehicle 1 and the landmark observations expected based on the reference map data to be detected. For example, landmarks may have disappeared or changed, or new landmarks may be detected.

[0184] A method can be applied to ensure that these are actual deviations of the real landmarks from the reference map data. In particular, artifacts from the detection of landmarks by vehicle 1 can be excluded. For this purpose, known methods can be used, in particular.

[0185] In a step S52, vehicle 1 generates update data based on the detected deviations between the reference map data and the actually recorded landmark observations. This update data includes information suitable for server 2 to update the reference map database in a further step S53. In this case, when repeating the process, for example, when repeatedly retrieving reference map data at a specific estimated position, new reference map data can now be provided that better reflect the actual features of the landmarks.

[0186] In this way, vehicle 1 can be used to re-survey the landmarks in the traffic environment. Furthermore, server 2 can receive update data from multiple sources, in particular multiple vehicles, in the same way and decide, for example using statistical methods, on an update of the reference map database. Furthermore, based on the update data, it can be determined whether a new survey of the landmarks should be performed in a specific traffic environment, for example, using a special surveying vehicle that enables particularly precise acquisition of the data for the reference map database.

[0187] In a further exemplary embodiment, it is provided that the reference map data is provided alternatively or additionally by a device other than the external server 2. For example, this can be a device of the vehicle 1 that allows retrieval of reference map data for landmark-based localization even without a data connection to the server 2. The method can be carried out analogously here, wherein the device of the vehicle 1 also accesses a reference map database and can update it using the update data. In a further exemplary embodiment, both the external server 2 and an on-board device are provided for retrieving reference map data, and the updating of the respective reference map database of the device and the server 2 can take place based on coordination between the two. List of reference symbols 1 Mobile Unit; Vehicle 2 External Server 3 Recording unit 3a laser scanner 3b Camera 3c GPS sensor 3n Additional sensor 4 Control unit 5 Interface 6 Computing unit 7 Accumulation unit 8 Segmentation unit 9 Landmark detection unit 10 tracking units 11 Output unit 12 Driver assistance system 13 Landmark; Delineator 14 Landmark; road boundary 15 Landmark; lane marking 16 Landmark; Guideline 17 Detection range 18 Roadway 18a, 18b lane S1a, ..., S1n Recording of data records S2a, ..., S2n landmark detection S3 Tracking and Fusion S4 Recording the estimated position S5 Proper motion estimation S6 Retrieval of reference card data S7 particle filter-based localization S8 Output of the position S21a Accumulation S22a Segmentation S23a Principal component analysis S24a classification S25a Generating landmark observations S31 Time synchronization S32 Distance calculations S33 Assigning and changing probabilities S34 Generating environment model data S40 Checking the initialization S41 Performing the initialization S42 Applying the motion model S43 Applying the sensor model S44 Assigning the weighting S45 smoothing S46 Output of the corrected pose S47 Performing resampling S50 Detecting the estimated position (vehicle) S51 Generating the reference map data (server) S52 Generating update data (vehicle) S53 Updating the reference card database (server)

Claims

[1] Method for detecting a traffic environment of a mobile unit (1), in which data records with data points are recorded by means of a recording unit (3), each data record being assigned a time point; landmark observations are determined for each of the recorded data sets, with landmark parameters and object classes being assigned to the landmark observations; the landmark observations are classified as point-based, line-based or polyline-based objects based on the object classes assigned to them; environmental model data is generated based on the specific landmark observations, wherein the environmental model data includes a description of the traffic environment; using the environment model data, distance calculations are carried out for landmark observations based on their object class and their landmark parameters, whereby distance values ​​are determined for the landmark observations, whereby the landmark observations are assigned to specific landmarks based on the distance values; whereby in the distance calculations - a Mahalanobis distance is determined for point-based objects, - for line-based objects, a modified Mahalanobis distance is determined based on the start and end positions and - for polyline-based objects, an area enclosed by the polyline-based objects is determined, whereby first a projection of line strings onto one another is carried out and the line strings are cut off in such a way that only those sections of the line strings which are opposite a section of another line string are further processed; probabilities are assigned to the environment model data, whereby the probabilities indicate the certainty with which a real landmark was detected and the assigned probabilities are changed based on the landmark observations, the environmental model data are updated using the probabilities assigned to the landmark observations; and the environment model data is output. [2] Method according to claim 1, characterized by that the specific landmark parameters for segments of the line-based or polyline-based feature class include a length parameter and a width parameter as well as a start position and an end position. [3] Method according to one of the preceding claims, characterized bythat the probabilities assigned to the environment model data are changed by means of a sequential probability ratio test, whereby the probabilities are used to check whether a landmark observation corresponds to an actual detection of a landmark (13, 14, 15, 16) of the traffic environment. [4] Method according to claim 3, characterized by , that a probability for a landmark observation is increased if another landmark observation is assigned to it, and the probability of a landmark observation is reduced if it is not confirmed by another landmark observation. [5] Method according to one of the preceding claims, characterized by that the landmark observations of the environmental model are accepted or rejected based on a comparison of the probabilities assigned to them with a threshold value. [6] Method according to one of the preceding claims, characterized by that the environment model data is assigned to landmarks and includes positions of the landmarks. [7] System for detecting a traffic environment of a mobile unit (1) with a recording unit (3) by which data records with data points can be recorded, wherein each data record can be assigned a time point; a landmark recognition unit (9), by means of which landmark observations can be determined for the recorded data sets, wherein landmark parameters and object classes can be assigned to the landmark observations; wherein the landmark observations can be classified as point-based, line-based or polyline-based objects based on the object classes assigned to them; a computing unit (6) by which environment model data can be generated on the basis of the determined landmark observations, wherein the environment model data comprise a description of the traffic environment; a tracking unit (10) by means of which distance calculations for landmark observations can be carried out on the basis of the environment model data based on their object class and their landmark parameters, wherein distance values ​​for the landmark observations can be determined; wherein in the distance calculations - a Mahalanobis distance can be determined for point-based objects, - for line-based objects, a modified Mahalanobis distance can be determined based on the start and end positions, and - for polyline-based objects, an area enclosed by the polyline-based objects can be determined, whereby first a projection of line strings onto each other is carried out and the line strings are cut off in such a way that only those sections of the line strings are further processed which are opposite a section of another line string; whereby the computing unit (6) is able to assign probabilities to the environment model data, the probabilities indicating the certainty with which a real landmark was detected, and the assigned probabilities being able to be changed based on the landmark observations; furthermore the environmental model data can be updated using the probabilities assigned to the landmark observations; and the environment model data can be output. [8] System according to claim 7, characterized bythat the detection unit (3) comprises at least two different sensor types (3a, 3b).

Images