Map credibility testing methods

By fusing sensor and map data and using Markov assumptions and Bayesian representations to update the existence probability of map elements, the problem of planning errors caused by outdated map data is solved, and accurate representation of the robot's environment and safe behavior planning are achieved.

CN116490747BActive Publication Date: 2026-11-10MERCEDES BENZ GRP +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202180066309.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-09-29
Filing Date
2021-06-16
Publication Date
2026-11-10
Estimated Expiration
2041-06-16

AI Technical Summary

Technical Problem

In the prior art, at least semi-autonomous robots rely on high-resolution maps (HD maps) for behavior planning, but outdated map data can lead to planning errors, and relying solely on sensor data may cause undesirable errors in traffic condition assessment.

Method used

By receiving sensor data and map data, and utilizing Markov assumptions and Bayesian representations, combined with random finite sets (RFS) and binary Bayesian filter methods, the existence probability and location of map elements are updated, and sensor and map data are fused to confirm the accuracy of map elements.

Benefits of technology

It provides a holistic, accurate, and effective representation of the robot's environment, ensuring the safety of behavior and trajectory planning and reducing uncertainties caused by outdated maps or sensor errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116490747B_ABST
    Figure CN116490747B_ABST
Patent Text Reader

Abstract

A map reliability testing method includes the following steps: receiving sensor data from a partially autonomous robot (F) forming an image of detected elements (S3, S4, S5), said at least one detected element (S3, S4, S5) being an environmental element of the partially autonomous robot (F) detected by an environmental sensor of the partially autonomous robot (F); receiving map data (Dk) forming an image of a map having at least one map element (A3, A4, A5), said at least one map element (A3, A4, A5) being drawn on a previously established map by the partially autonomous robot (F). The robot (F) is exposed to environmental elements; receives localization data (Dl) indicating the position of at least partially autonomous robot (F) on a map; determines data uncertainty, including sensor data uncertainty, map data uncertainty, and / or localization data uncertainty; initializes the existence probability (P) of at least one map element (A3, A4, A5) with initial values; and updates the existence probability (P) of at least one map element (A3, A4, A5) using map data (Dk), sensor data (Ds), localization data (Dl), and data uncertainty.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a method for testing the credibility of maps. Background Technology

[0002] Currently, at least semi-autonomous robots, and especially partially autonomous vehicles, rely heavily on high-resolution maps (so-called HD maps), particularly for planning their future behavior. Relying solely on sensor data from at least semi-autonomous robots can lead to unintended errors in traffic condition assessments.

[0003] However, if the map data is outdated, the future behavior of at least semi-autonomous robots cannot be planned without errors.

[0004] Therefore, a method for verifying map data is desired. Summary of the Invention

[0005] Embodiments of the present invention provide a map verification method according to the independent claims. Useful embodiments of the present invention will be apparent from the dependent claims, the specification, and the drawings.

[0006] According to one aspect of the present invention, a map verification method or a card credibility check method includes the following steps: Receiving sensor data of at least a semi-autonomous robot depicting at least one detected element, wherein the at least one detected element represents an environmental element of the at least semi-autonomous robot detected by environmental sensors of the at least semi-autonomous robot. Receiving map data depicting a map having at least one map element, wherein the at least one map element represents an environmental element of the at least semi-autonomous robot drawn on a predetermined map. Determining positioning data based on the received sensor data, wherein the positioning data indicates the position of the at least semi-autonomous robot on the map. Determining data uncertainty, wherein the data uncertainty includes sensor data uncertainty, map data uncertainty, and / or positioning data uncertainty. Initializing the presence probability of at least one map element with initial values. Updating the presence probability of at least one map element using map data, sensor data, positioning data, and data uncertainty.

[0007] Preferably, the sensor data of the at least semi-autonomous robot includes online sensor data detected by the sensors of the at least semi-autonomous robot. Preferably, the sensor data includes camera data, LiDAR data, radar data, and / or GPS data.

[0008] The term “map element” as used in this article specifically includes traffic signs such as traffic signs or traffic lights, as well as road markings.

[0009] In general, a semi-autonomous robot can be a vehicle that is at least partially autonomous. Alternatively, a semi-autonomous robot can be another mobile robot, such as one that moves by flying, swimming, diving, or walking. For example, a mobile robot can also be a semi-autonomous lawnmower or a semi-autonomous cleaning robot.

[0010] Preferably, initializing the existence probability includes determining an initial value for the existence probability of a map element, which indicates a provisional existence probability assigned to the map element in the absence of sensor data.

[0011] Preferably, the positioning data includes GPS data.

[0012] Preferably, the location data is determined using received sensor data and received map data. For example, location data is determined by matching map data with sensor data, particularly by means of GPS data. Thus, the location data indicates at least an estimated position of the semi-autonomous robot on the map.

[0013] Therefore, the map verification method uses the so-called Markov assumption, according to which the actual state depends only on previous states and current sensor data, and not on the complete history of states. In this way, the map verification method can achieve perhaps the simplest computational load.

[0014] Preferably, determining data uncertainty includes representing sensor data, map data, and location data in a probabilistic manner.

[0015] Preferably, the probability representation includes Bayesian representation.

[0016] The probabilistic representation of sensor data takes into account the spatial uncertainty of sensor measurements, the interference rate of sensor measurements, the detection probability of (one or more) sensors for map elements, and / or depends on the hit and miss concentrations of the formula used:

[0017] Disturbances (also known as clutter) include sensor measurements, i.e., sensor data that is not determined by real objects, such as map elements, like ghost measurements.

[0018] The clutter rate describes the number of clutter measurements per time step, and when clutter measurements occur independently, the number of clutter measurements is typically modeled using a Poisson distribution.

[0019] Preferably, the map verification method is performed in at least a semi-autonomous robot; in other words, it is performed online. The map verification method uses not only map data and sensor data, but also the localization of the at least semi-autonomous robot, i.e., its estimated position relative to the map. The task is to identify or remove map elements, and thus detect possible discrepancies between the map data and the sensor data (i.e., the environment of the at least semi-autonomous robot).

[0020] The proper operation of map verification methods requires not only accurate positioning but also precise sensor calibration. This is because discrepancies between detected map data and sensor data can be caused not only by incorrect or outdated maps but also by poor positioning and improperly calibrated or faulty sensors.

[0021] Preferably, the co-existence probability of map elements is calculated based on different sensors. Alternatively, the individual existence probability of map elements is calculated for each sensor or different combinations of sensors.

[0022] Preferably, the sensor data includes data from untracked sensor measurements: unlike the tracked elements that are filtered over time, errors from sensor measurements have little or no correlation with time. Measurements with time-independent errors are easier to model probabilistically, and such models are also easier to examine as part of internal validity checks. There is no measurement loss, and the map verification method has access to all available information, especially including perturbation measurements that have already been filtered out by the tracking algorithm.

[0023] Updating the existence probability falls under the concept of element fusion or object fusion. In other words, it involves calculating the existence probability of directly observable map elements by fusing map data and sensor data. This can be done individually for each sensor, but it is also possible to calculate the joint existence probability based on all available sensor data or any combination of sensors at once.

[0024] In addition to the existence probability, the updated position of map elements is preferably calculated. This is done intrinsically via a (multi) Bernoulli filter in the Random Finite Set Method (RFS method), and can also be added as an additional Kalman filter via the Log-Binary Bayesian Filter Method (logit method) if needed.

[0025] The probability representation of the input data is used to calculate the probability of updating the map elements in the horizon for each visible map element in each time step.

[0026] In this way, improved behavior and trajectory planning regarding safety are provided.

[0027] In this way, a holistic, accurate, and effective representation of the environment of at least a semi-autonomous robot is provided based on fused sensor and map data.

[0028] According to a preferred embodiment, the map verification method includes the following steps: Projecting at least one map element into the sensor space of an environmental sensor.

[0029] Map elements, as part of the map horizon, are projected onto all relevant (e.g., forward-facing) sensors (i.e., their sensor space). This projection is generally straightforward, while the reverse projection from sensor coordinates to map coordinates is often more complex or even undefined, as in the case of a monocular camera. The map horizon is a subset of the total map, consisting of map elements at least near the semi-autonomous robot. This map horizon includes at least the field of view of all relevant sensors, but is typically larger.

[0030] According to a preferred embodiment, the map verification method includes the following steps: assigning at least one map element to at least one element to be detected.

[0031] In the assignment step, the optimal global allocation of a predetermined number of sensor data (i.e., sensor measurements) depicting elements in the sensor space is computed using Murty's ranking assignment algorithm and, for example, the Hungarian method as its underlying principle. The ranking assignment can be based on, for example, a multi-object measurement model derived from RFS theory, which captures not only the spatial uncertainty of sensor measurements but also the perturbation rate, perturbation strength, and detection probability. Specifically, (multi)Bernoulli filters are capable of handling multiple associations based on their respective probability weights computed in the association step. In particular, these associations also consider the possibilities of missed sensing (map elements not assigned to any sensing) and interfering sensing (sensings not assigned to any map elements).

[0032] Preferably, the association between the detected element and the map element is synchronized only once within the system and among all modules that depend on this information. Otherwise, the map verification method would be separated from other functional modules, which might perform completely different associations and use them without checking the correctness of these potentially incorrect associations. The association of measurements depicting map objects requires location as input. The association relies on predicted location estimates from previous time steps, and the provided association is used to update the upcoming steps of the location module.

[0033] According to a preferred embodiment, the map verification method includes the following steps: assessing the probability of the existence of at least one map element, wherein the assessment includes one of the following: confirming the map element, refuting the map element, identifying a potential new map element, and being unable to provide a description.

[0034] If sensor data cannot be assigned to any existing map element, potential new map elements are identified. In other words, the probability that a measurement (i.e., the corresponding sensor data) cannot be assigned to an existing map element is determined. Therefore, potential new map elements are reported, or new map elements are initialized directly.

[0035] According to a preferred embodiment, updating the existence probability includes the random finite set (RFS) method or the logit method.

[0036] The RFS method refers to a sensor model that uses random finite set theory and is based on spatial uncertainty, perturbation rate, perturbation strength, and detection probability.

[0037] The logit method refers to the log-odds implementation using a binary Bayesian filter, based on the hit rate and miss rate of the sensor model.

[0038] By applying (multi) Bernoulli filters, the RFS method can provide updated locations of map elements in addition to existence probabilities. In this case, the logit method must also be combined with a Kalman filter to provide location updates.

[0039] Using all this information, the existence probability of previous map elements can be updated using either the RFS or Logit method. The existence probability is based on collected but uncertain sensor data to indicate whether a map element still exists. Map elements that are not visible at all are not updated and therefore retain their existence probability from previous time steps. Updating the existence probability of map elements, and preferably updating the location of map elements, occurs at each time step where one set of sensor measurements is available. Both RFS and Logit methods follow the Markov assumption, in which the actual state of the updated map element depends only on the current measurement and its previous states, and not on the complete history of the map element's states.

[0040] Updating the existence probability can be accomplished either using a (multi) Bernoulli filter and an RFS-based measurement model (which also uses sensor data as a byproduct to update the location of map objects), or using a binary Bayesian filter with log-likelihood and corresponding hit / miss rates.

[0041] The Dobernouri filter is based on RFS theory and takes into account the spatial uncertainty of sensor measurements, sensor perturbation rate / intensity, and detection probability of map elements.

[0042] Binary Bayesian filters, also known as logit or log-odds methods, model the hits and misses of elements with specific probabilities. Measurements near a map element are considered hits, and any measurement near a particular map element is not considered a miss if that particular map element is visible.

[0043] For point-like objects or objects that sparsely extend relative to the sensor space, the described method can be applied directly, such as traffic lights, traffic signs, dashed lane markings, utility poles, tree trunks, Botts dots, etc. If solid lane markings (e.g., boundaries) are too long and therefore significantly exceed the sensor's field of view, it may be necessary to divide them into smaller segments. The existence of these smaller segments can then be checked using the proposed method. Thus, the probability of existence is assigned to each segment of a (continuous) fixed lane boundary.

[0044] As a byproduct of using the RFS method, which relies on (multi) Bernoulli filters, the probability of measurements that do not belong to any existing set of objects can be calculated. This information is a perfect indicator of potential new map elements. Therefore, additional measurements with a high probability of not belonging to any known map elements are forwarded to allow for the justification of new map elements. These measurements can also be used to create new map elements and update them in the following steps of the map validation method.

[0045] According to the preferred embodiment, the existence probability is initialized with an initial value of 50%.

[0046] Preferably, the initial values ​​are predetermined. More preferably, the initial values ​​are dynamically determined for each map element. For example, to determine the initial values, a predetermined trend of the probability of existence is determined based on the characteristics of the map element and / or map data. For example, if it is assumed that road signs are generally replaced more frequently than traffic lights, then map elements of the road sign type are initialized with a lower initial probability than those of traffic light type map elements.

[0047] According to a preferred embodiment, the existence probability of at least one map element is repeatedly updated at regular time intervals.

[0048] Preferably, the time interval is predetermined. Furthermore, preferably, the time interval depends on the detection rate of at least one environmental sensor of the semi-autonomous robot. In other words, the previous presence probability of at least one map element is updated as soon as new sensor data becomes available. For example, the environmental sensors of the semi-autonomous robot include cameras providing 25 frames per second. Therefore, the previous presence probability of at least one map element is updated 25 times per second.

[0049] According to a preferred embodiment, the map verification method includes the following steps: determining the visibility of map elements, wherein the visibility of map elements is determined using the field of view of an environmental sensor and the occlusion of map elements; and using the visibility of map elements to determine the detection probability.

[0050] The visibility of map elements is checked not only by the sensor’s field of view, but also by the occlusion of the corresponding map elements.

[0051] Preferably, the detection probability is determined using the true rate of the sensor.

[0052] Furthermore, visibility takes into account the uncertainties of sensor measurements and map elements, as well as the uncertainties of positioning and calibration information propagated via error. Visibility information can then be integrated into the detection probability. The detection probability captures not only the visibility of map objects but also typically the probability that a sensor or detection algorithm will produce a corresponding measurement. Therefore, the detection probability of an invisible object is zero, while the detection probability of a fully visible object is usually not 1. This is because sensors typically cannot detect all visible objects, but only a certain proportion of them (see True Positive Result Rate).

[0053] The visibility of map elements is estimated individually for each sensor, specifically by examining the sensor's perceptible field of view and any occlusion from other objects in front of the map element, such as by examining the line of sight from the sensor origin to the corresponding map element. This can be done, for example, using a stixel representation from a stereo camera system that provides a 3D representation of the environment. This representation can then be used to check if such a stixel is located in front of a map element, thus hiding it. An alternative is to use a labeled pixel image from a monocular camera, which can be specialized for all map elements on the ground. The basic idea is to check if pixels in front of the expected map element are classified as objects, such as cars, pedestrians, etc., to know if they are hiding map elements on the ground behind them, such as lane markings.

[0054] Then, the detection probability of a map element is derived not only from its estimated visibility but also from the true rate of the sensor for that specific map element type. The detection probability indicates the likelihood that a map element will generate a corresponding sensor reading. This is closely related to the hit rate in the logit representation and is directly integrated into the measurement model of the RFS method. For completely invisible map elements, the detection probability is zero, while for fully visible objects, the detection probability is less than or equal to 1. For fully visible map elements and sensors with a true rate of 1, the detection probability is only 1; that is, in this case, the sensor does not provide any false negative measurements.

[0055] The true rate indicates the percentage of detected map elements relative to the total number of map elements present in the environment; in other words, it describes the ratio of true measurements to the sum of true and false negative measurements.

[0056] Detection probability includes the probability that existing map elements will generate corresponding measurements (i.e., sensor data). This includes the visibility of map elements and the accuracy of environmental sensors.

[0057] According to a preferred embodiment, data uncertainty is used to determine the visibility of map elements.

[0058] According to a preferred embodiment, the presence probability of at least one map element whose detection probability is lower than a predetermined threshold is not updated.

[0059] According to a preferred embodiment, the map verification method includes the following steps: Verifying the validity of the existence probability.

[0060] To verify the effectiveness of the map verification method itself, the stochastic assumptions used to calculate the existence probability are examined in at least a semi-autonomous robot under an internal credibility or validity check. This validity check reports statistically significant deviations of the assumed parameters (e.g., spatial uncertainty, perturbation rate, and detection probability in RFS-based methods, or hit and miss rates in logit methods) from those estimated online at a predetermined significance level. Additionally, error propagation from the map to the sensor space requires linearization of a latent nonlinear function; as a validity check, the strong nonlinearity of this latent nonlinear function in the relevant domain is also examined.

[0061] The random and algorithmic hypotheses used to update the existence probability are validated online as an internal validity or consistency check. Preferably, the consistency of the measurement model of the environmental sensor is checked. Relevant parameters of the measurement model are estimated online, such as the spatial uncertainty of the sensor measurement, the sensor perturbation rate, and the detection probability when using an RFS-based formula. Subsequently, a validity check verifies whether the hypothesis initially made to calculate the existence probability is within the confidence intervals of these online estimated parameters. For this purpose, statistical hypothesis testing is preferably performed. Preferably, the parameters are estimated online. However, again, to avoid self-fulfilling predictions, they are not adjusted accordingly. Therefore, the estimated parameters are checked only against the hypothesized parameters to detect random deviations between the two, and a confidence check itself is no longer reliable, reported to another module. This is a form of self-evaluation of the confidence check itself.

[0062] According to a preferred embodiment, sensor data from different sensors of at least a semi-autonomous robot are compared to verify the validity of the existence probability.

[0063] According to another aspect of the invention, the map verification system is configured to perform the map verification method specified herein.

[0064] According to another aspect of the invention, a method for controlling at least a semi-autonomous robot includes the following steps: performing a map verification method as specified herein to determine the probability of presence of at least one map element; determining a robot trajectory using sensor data, map data, localization data, and the probability of presence of at least one map element; and controlling the at least semi-autonomous robot based on the determined robot trajectory.

[0065] In this way, the map verification method allows behavior and trajectory planning to safely use map data by confirming relevant portions of the map with sensor data (particularly online sensor data of the at least semi-autonomous robot) before relying on their presence in the environment of the robot. Based on the output of the map verification method, the behavior planning module will know which map elements have been confirmed with the sensor data, which map elements are unknown (e.g., due to occlusion), and which map elements are no longer present. Therefore, the map verification method provides a holistic view, accurate, and most importantly, effective representation of the environment of the at least semi-autonomous robot based on fused sensor and map data.

[0066] The following examples illustrate behavioral planning:

[0067] Example 1

[0068] When a vehicle approaches an intersection where the traffic light is missing, it will not be hidden if the traffic light still exists. As the vehicle approaches the missing traffic light, the probability of its presence will gradually decrease from 50% to 0%. Behavior generation will cause the vehicle to slow down quite early in response to the reduced probability of presence and eventually stop safely when the traffic light on the map is critical for safety, thus avoiding entering the intersection while relying on outdated map data.

[0069] Example 2

[0070] A vehicle is approaching an intersection where the traffic lights are hidden. Behavior generation will return to slowing the vehicle down and / or, if possible, driving around the object where the traffic lights are hidden. The idea behind this is to gather more information about the presence of the traffic lights. If the traffic lights are not visible and if they are critical to safety, the vehicle will stop before the intersection.

[0071] Example 3

[0072] The vehicle is approaching an intersection with clearly visible traffic lights. As the vehicle approaches the intersection, the probability of the traffic lights being present gradually increases from 50% to 100%. Because the vehicle can now perceive the traffic lights and their condition, it does not need to slow down to safely cross the intersection.

[0073] According to a preferred embodiment, a method for controlling at least a semi-autonomous robot includes the following steps: determining a control mode using sensor data, map data, positioning data, and the presence probability of at least one map element, and controlling the at least semi-autonomous robot based on the determined control mode.

[0074] Preferably, the control modes include predetermined behavioral characteristics of at least the semi-autonomous robot. For example, the control modes include a "normal driving" mode, a "preventive safety" mode, and / or a "safe stop" mode. Given the same input data (i.e., sensor data, map data, and / or positioning data), different controls are applied to the at least semi-autonomous robot depending on the control mode.

[0075] Preferably, the computer program includes instructions that, when executed by a computer, cause the computer program to perform the map verification method specified herein.

[0076] Preferably, the computer program described herein is stored on a machine-readable storage medium.

[0077] Advantageously, map data including at least one map element is received from a remote server, and the results of the map verification method are advantageously transmitted to the remote server and used there as the basis for determining whether to trigger an update of at least one map element or to trigger a map retest at the location of at least one map element. The results of the map verification method are, in particular, the probability of the existence of an update of at least one map element, or information derived therefrom, indicating whether the existence of at least one map element is confirmed, refuted, or no statement can be made regarding its existence. Attached Figure Description

[0078] Further measures to improve the invention will now be described in more detail with reference to the accompanying drawings and preferred embodiments.

[0079] Examples of implementation methods

[0080] It shows:

[0081] Figure 1a It is a map verification method based on the first implementation method within the first time step;

[0082] Figure 1b It is a map verification method based on the first embodiment in the second time step;

[0083] Figure 1c It is a map verification method based on the first implementation method in the third time step;

[0084] Figure 2 This is a schematic diagram of the map verification method;

[0085] Figure 3a It is a map verification method according to the second embodiment within the first time step;

[0086] Figure 3b It is a map verification method according to the second embodiment in the second time step;

[0087] Figure 3c The map verification method according to the second embodiment is performed at the third time step; and

[0088] Figure 4 It is a map verification system. Detailed Implementation

[0089] Figure 1a The first traffic situation V1 is shown in the first time step. At least a semi-autonomous robot (in this case, a vehicle with at least partial autonomous driving) is moving towards the intersection. Two map elements, a first traffic light A1 and a second traffic light A2, are obtained from map data of a high-resolution (HD) map. However, in this case, the first traffic light A1 has been removed due to construction work. In this respect, the map data regarding the first traffic light A1 is outdated. According to the map verification method, for the first traffic light A1 and the second traffic light A2, the first presence probability P1 and the second presence probability P2 are initialized with a value of 50%. In this first time step, vehicle F is still relatively far from the intersection. The sensor data provided by the environmental sensors of vehicle F does not contain any elements detected by either the first traffic light A1 or the second traffic light A2. In the first time step, a confidence check is performed to update the first presence probability P1 and the second presence probability P2. In the reliability check, based on the internal statistics and positioning data of vehicle F, it is determined that vehicle F is still far enough from the intersection that its environmental sensors cannot detect either the first traffic light A1 or the second traffic light A2. In other words, the first traffic light A1 and the second traffic light A2 are not in the field of view of vehicle F's environmental sensors. Therefore, based on the sensor data, it is determined that no further statement can be made regarding the first probability of presence P1 and the second probability of presence P2. In this respect, the first probability of presence P1 remains at 50%, and the second probability of presence P2 remains at 50%. To control the vehicle, which is at least partially automated, based on the sufficient distance of vehicle F to the intersection, it is decided to continue driving at a normal speed. In this case, the vehicle's control mode is "normal driving".

[0090] Figure 1bThe first traffic condition V1 is shown in the second time step. The at least partially automated vehicle F is closer to the intersection than in the first time step. In the second time step, a credibility check is performed to update the first presence probability P1 and the second presence probability P2. In the credibility check, based on the internal statistics and location data of vehicle F, it is determined that vehicle F is close enough to the intersection that its environmental sensors should be able to detect the first traffic light A1 and the second traffic light A2. In other words, the first traffic light A1 and the second traffic light A2 are within the line of sight of vehicle F's environmental sensors. Therefore, based on the sensor data, it is determined that a statement can be made regarding the first presence probability P1 and the second presence probability P2. Since the indicated first traffic light A1 no longer exists, vehicle F's environmental sensors will not detect any element at the expected location in the map data. Conversely, the second traffic light A2 remains at the expected location in the map data. In this respect, the first presence probability P1 decreases from 50% to 35%, and the second presence probability P2 increases from 50% to 70%. In order to control the vehicle, which is at least partially automated, based on a first probability of existence P1 and a second probability of existence P2, it is decided to slightly decelerate the vehicle F to allow more time to measure the environment in order to make further descriptions, particularly regarding the first traffic light A1. The control mode of the vehicle F changes from "normal driving" to "preventive safety".

[0091] In this scenario, a discrepancy was detected between the sensor data and map data regarding the first traffic light A1 during the reliability check. If the environmental sensors of vehicle F also detect the first traffic light A1, the first probability of presence P1 will increase from 50% to 70%, and the control mode of vehicle F will remain unchanged under "normal driving".

[0092] Figure 1cThe first traffic condition V1 is shown in the third time step. The at least partially automated vehicle F is closer to the intersection than in the second time step. In the third time step, a credibility check is performed to update the first presence probability P1 and the second presence probability P2. In the credibility check, based on the internal statistics and location data of vehicle F, it is determined that vehicle F is still close enough to the intersection that its environmental sensors should be able to detect the first traffic light A1 and the second traffic light A2. In other words, the first traffic light A1 and the second traffic light A2 are within the field of view of vehicle F's environmental sensors. Therefore, based on the sensor data, it is determined that a statement can be made regarding the first presence probability P1 and the second presence probability P2. Since the indicated first traffic light A1 no longer exists, vehicle F's environmental sensors continue to fail to detect any element at the expected location in the map data. Conversely, the second traffic light A2 remains at the expected location from the map data. In this respect, the first presence probability P1 decreases from 35% to 5%, and the second presence probability P2 increases from 70% to 99%. Based on the first probability of existence P1 and the second probability of existence P2, it is decided to control at least partially automated vehicles to ensure that vehicle F comes to a safe stop before the intersection, because the possibility of a potential emergency situation is high due to the absence of the first traffic light A1. The control mode of vehicle F here changes from "preventative safety" to "safe stop". In this case, remote operation of vehicle F is requested from the monitoring center, which then manually resolves the traffic situation for vehicle F. Additionally, map data updates or intersection update measurements are initiated.

[0093] Figure 2 A map verification method comprising the following steps is shown. In a first step S10, sensor data depicting a vehicle with at least one detected element is received, the at least one detected element representing environmental elements of the vehicle detected by the vehicle's environmental sensors. In a second step S20, map data depicting a map having at least one map element representing environmental elements of the vehicle drawn on a predetermined map is received. In a third step S30, positioning data indicating the vehicle's position on the map is determined based on the received sensor data. In a fourth step S40, data uncertainty is determined, including sensor data uncertainty, map data uncertainty, and / or positioning data uncertainty. In a fifth step S50, the presence probability of at least one map element is initialized. In a sixth step S60, the presence probability of at least one map element is updated using the map data, sensor data, positioning data, and data uncertainty.

[0094] Figure 3a The second traffic condition V2 is displayed at the first time step.

[0095] At least partially automated vehicle F is located near the intersection. From the high-resolution (HD) map data, three map elements can be seen: traffic light A3 (third), traffic light A4 (fourth), and traffic light A5 (fifth). In this case, traffic lights A3, A4, and A5 are actually still present. In this respect, the map data is valid.

[0096] However, only the third detected element S3, which can be assigned to the third traffic light A3, and the fourth detected element S4, which can be assigned to the fourth traffic light A4, are generated from sensor data.

[0097] According to the map verification method, for the third traffic light A3, the fourth traffic light A4, and the fifth traffic light A5, the probability of their existence P3, P4, and P5 is initialized with a value of 50%. In the first time step, a reliability check is performed to update the probability of their existence P3, P4, and P5. In the reliability check, based on the internal statistics and location data of vehicle F, it is determined that vehicle F is close enough to the intersection that its environmental sensors should be able to detect the third traffic light A3, the fourth traffic light A4, and the fifth traffic light A5. However, sensor data from vehicle F also reveals the presence of trucks L and LKW near vehicle F, within the line of sight between vehicle F's environmental sensors and the fifth traffic light A5. In other words, the third traffic light A3 and the fourth traffic light A4 are in the field of view of vehicle F's environmental sensors, while the fifth traffic light A5 is not. Therefore, it is determined that no further statement can be made regarding the probability of the fifth traffic light P5, but statements can be made regarding the probability of the third traffic light P3 and the fourth traffic light P4. In this regard, during the first time step, the probability of the fifth existence, P5, remains at 50%, while the probabilities of the third existence, P3, and the fourth existence, P4, increase from 50% to 77.1%.

[0098] Figure 3b The second traffic condition V2 is shown at the second time step.

[0099] Based on the sensor data, there are only the third detected element S3 that can be assigned to the third traffic light A3 and the fourth detected element S4 that can be assigned to the fourth traffic light A4.

[0100] In the second time step, a credibility check is performed to update the third presence probability P3, the fourth presence probability P4, and the fifth presence probability P5. In the credibility check, based on the internal statistics and location data of vehicle F, it is determined that vehicle F is still sufficiently close to the intersection that its environmental sensors should be able to detect the third traffic light A3, the fourth traffic light A4, and the fifth traffic light A5. However, vehicle F's sensor data continues to reveal the presence of trucks L and LKW near vehicle F, which are within the field of view between vehicle F's environmental sensors and the fifth traffic light A5. In other words, furthermore, the third traffic light A3 and the fourth traffic light A4 are within the field of view of vehicle F's environmental sensors, while the fifth traffic light A5 is not. Therefore, it is determined that no further statement can be made regarding the fifth presence probability P5, but statements can be made regarding the third presence probability P3 and the fourth presence probability P4. In this respect, in the second time step, the fifth presence probability P5 remains at 50%, while the third presence probability P3 and the fourth presence probability P4 each increase from 77.1% to 99.5%.

[0101] Figure 3c The second traffic condition V2 is shown in the third time step.

[0102] The sensor data now generates a third detected element S3 that can be assigned to the third traffic light A3, a fourth detected element S4 that can be assigned to the fourth traffic light A4, and a fifth detected element S5 that can be assigned to the fifth traffic light A5.

[0103] In the third time step, a credibility check is performed to update the third presence probability P3, the fourth presence probability P4, and the fifth presence probability P5. In the credibility check, based on the internal statistics and location data of vehicle F, it is determined that vehicle F is still sufficiently close to the intersection so that its environmental sensors should be able to detect the third traffic light A3, the fourth traffic light A4, and the fifth traffic light A5. Furthermore, the sensor data of vehicle F reveals that truck L near vehicle F is no longer within the field of view between vehicle F's environmental sensors and the fifth traffic light A5. In other words, the third traffic light A3, the fourth traffic light A4, and the fifth traffic light are within the visual range of vehicle F's environmental sensors. Therefore, it is determined that statements regarding the fifth presence probability P5, the third presence probability P3, and the fourth presence probability P4 can be made. In this respect, in the third time step, the fifth presence probability P5 increases from 50% to 69.5%, and the third and fourth presence probabilities P3 and P4 increase from 99.5% to 100%.

[0104] Figure 4A map verification system 10, comprising an object mapping unit 11 and an object fusion unit 12, is shown. The map verification system 10 is coupled as input to a sensor system 20, a positioning system 30, and a map system 40. Furthermore, the map verification system 10 is coupled as output to a behavior planning system 50.

[0105] Sensor system 20 provides sensor data Ds. Map system 40 provides map data Dk. Positioning system 30 determines positioning data Dl based on sensor data Ds and map data Dk. Sensor data Ds, map data Dk, and positioning data Dl are provided to map verification system 10, and in particular to object allocation unit 11.

[0106] Object mapping unit 11 is specifically designed for each sensor type and is instantiated once per sensor. Object mapping unit 11 is configured to project map elements onto the corresponding sensor space, e.g., onto the camera coordinate system, and is configured to estimate the visibility of map elements in the sensor space, e.g., based on bar pixels that might obstruct the view of these map elements. Object assignment unit 11 is also configured to compute k optimal associations using probabilistic formulas and ranking assignment algorithms (e.g., using the Murty algorithm) based on sensor data. Object assignment is outsourced from the actual map reliability check, as it is not map-specific, and its results can be used by other modules, such as localization. Object mapping unit 11 is thus configured to project map elements into the sensor space, or in other words, project map elements into the sensor coordinate system, to determine, specifically estimate, the visibility of map elements, and assign sensor data (i.e., sensor measurements) to depict map elements in the sensor space.

[0107] The object fusion unit 12 is configured to update the presence probability of map elements and optionally also to update the location of map elements. Furthermore, the object fusion unit 12 is configured to identify sensor data (i.e., sensor measurements) that have not been assigned to any existing map elements and thus potentially identify new map elements. The object fusion unit 12 is configured to check the correctness of the algorithm / random hypothesis, i.e., to check the validity of the presence probability.

[0108] Therefore, the map verification system 10 provides the behavior planning system 50 with map elements having an existence probability P, unassigned measurements Mu, and validity check results V. Thus, the behavior planning system 50 controls the behavior of the at least partially automated vehicle based on the provided data.

Claims

1. A map verification method, comprising the following steps: Receive (S10) sensor data of at least a semi-autonomous robot (F) for describing at least one detected element (S3, S4, S5), wherein the at least one detected element (S3, S4, S5) represents an environmental element of the at least semi-autonomous robot (F) detected by the environmental sensors of the at least semi-autonomous robot (F). Receive (S20) map data (Dk) depicting a map having at least one map element (A3, A4, A5), wherein the at least one map element (A3, A4, A5) represents an environmental element of the at least semi-autonomous robot (F) drawn on a predetermined map; Receive (S30) positioning data (Dl), the positioning data (Dl) indicating the position of the at least semi-autonomous robot (F) on the map; Determine (S40) the data uncertainty, wherein the data uncertainty includes sensor data uncertainty, map data uncertainty and / or positioning data uncertainty; The at least one map element (A3, A4, A5) is projected into the sensor space of the environmental sensor; The visibility of map elements (A3, A4, A5) is determined using the field of view of the environmental sensor and the occlusion of the map elements (A3, A4, A5). The detection probability is determined using the visibility of the map elements (A3, A4, A5); The existence probability (P) of at least one map element (A3, A4, A5) is initialized with an initial value (S50); The existence probability (P) of at least one map element (A3, A4, A5) is updated (S60) using the map data (Dk), the sensor data (Ds), the positioning data (Dl), and the data uncertainty; wherein the existence probability (P3, P4, P5) of at least one map element (A3, A4, A5) with a detection probability lower than a predetermined threshold is not updated; wherein the updated existence probability of at least one map element indicates whether the existence of at least one map element is confirmed, refuted, or no statement can be made regarding its existence.

2. The map verification method according to claim 1, comprising the following steps: Assign the at least one map element (A3, A4, A5) to the at least one detected element (S3, S4, S5).

3. The map verification method according to claim 1 or 2, comprising the following steps: Evaluate the probability of existence (P3, P4, P5) of the at least one map element (A3, A4, A5). The evaluation includes one of the following: confirming the map element, refuting the map element, identifying a potential new map element, and being unable to provide an explanation.

4. The map verification method according to claim 1 or 2, in, Updating the existence probabilities (P3, P4, P5) includes the Random Finite Set (RFS) method or the logit method.

5. The map verification method according to claim 1 or 2, in, Initialize the existence probabilities (P3, P4, P5) with an initial value of 50%.

6. The map verification method according to claim 1 or 2, in, The existence probability (P3, P4, P5) of at least one map element (A3, A4, A5) is repeatedly updated at regular time intervals.

7. The map verification method according to claim 1 or 2, in, The data uncertainty is used to determine the visibility of the map elements (A3, A4, A5).

8. The map verification method according to claim 1 or 2, comprising: Verify the validity of the stated existence probabilities (P3, P4, P5).

9. The map verification method according to claim 8, in, Sensor data (Ds) from different sensors of the at least semi-autonomous robot (F) are compared with each other to verify the validity of the existence probabilities (P3, P4, P5).

10. A map verification system (10) configured to perform the map verification method according to any one of claims 1 to 9.

11. A method for controlling at least a semi-autonomous robot, comprising the following steps: Perform the map verification method according to any one of claims 1 to 9 to determine the probability of the existence of at least one map element; The robot trajectory is determined using sensor data (Ds), map data (Dk), positioning data (Dl), and the presence probability (P3, P4, P5) of at least one map element (A3, A4, A5). The robot, at least semi-autonomous, is controlled based on the determined robot trajectory.

12. The method according to claim 11, The control mode is determined using the sensor data (Ds), the map data (Dk), the positioning data (Dl), and the presence probability (P3, P4, P5) of at least one map element (A3, A4, A5). The robot, which is at least semi-autonomous, is controlled based on the determined control mode.

Citation Information

Patent Citations

  • System for generating confidence values ​​in the backend

    DE102018204501B3