Localization methods using different sensors

Abstract

Calibration procedure for a vehicle that has sensor data, the procedure encompassing: Generating (S210) several particles, each indicating an estimated motion state of the vehicle, Superimposing (S220) the motion of the generated particles with noise, whereby a running average is formed for each particle, Weights (S230) of the superimposed particles according to a probability of agreement, Periodic re-sampling (S240) of the particles according to their weights, Comparing the current mean with a threshold; and if the current mean is greater than the threshold: Counteracting miscalibration.

Description

[Field of invention]

[0001] The invention relates to a method for locating a vehicle with different sensors, and to a vehicle that performs such a method. [Background]

[0002] Reliable and error-free vehicle localization is essential for many applications, particularly for driver assistance systems and even automated driving. Localization using satellite-based navigation systems alone is too imprecise for this purpose. Localization can be improved by combining it with additional sensor data, such as odometry sensors. However, wear and tear can introduce errors into the system; for example, if the wheel circumference changes by an unknown amount, wheel sensor-based localization will systematically deliver incorrect values.

[0003] To avoid such errors, manual intervention is often required; for example, manual calibration is necessary to measure the exact wheel circumference or the exact movement of the wheel as it rolls, in order to make corrections based on this.

[0004] Calibration parameters can also be obtained during vehicle operation by selecting and optimizing the most probable value of a calibration parameter, thus providing the best possible explanation for currently measured sensor data. However, such calibration requires numerous calculation steps to determine the optimal calibration parameters during vehicle operation.

[0005] Such an approach is described in the article by Kümmerle, R., Grisetti, G., & Burgard, W.: “Simultaneous calibration, localization, and mapping” 2011 IEEE / RSJ International Conference on Intelligent Robots and Systems (pp. 3716-3721). The position of a robot is determined by odometry and a laser-based method. Simultaneous calibration and localization are performed using hyper-graphs and a maximum likelihood configuration.

[0006] Particle filter-based localization techniques offer advantages, such as multimodality and the consideration of nonlinearities in the state space and measurement model, but they have the disadvantage that the computational effort increases sharply with the number of dimensions of the state space. Therefore, the known methods and their implementations are very computationally intensive, making their application in road traffic either impossible or extremely limited.

[0007] Document DE 10 2016 214 027 A1 discloses a method and a system for detecting landmarks in a traffic environment of a mobile unit.

[0008] Document US 2014 / 0372026A1 discloses a navigation module and a method for providing a GNSS navigation solution.

[0009] Document DE 10 2011 120 497 A1 discloses systems and methods for precise vehicle positioning within a lane.

[0010] Document DE 10 2019 126 431 A1 discloses a method and a device for locating a vehicle. [Concept of the invention]

[0011] It is therefore an object of the present invention to provide a localization method that requires less computing power and can also be used at the typically high speeds and complex situations in road traffic.

[0012] The problem is solved by the subject matter of the independent claims.

[0013] Preferred embodiments are described in the dependent claims.

[0014] In one embodiment of the present invention, a method comprises generating several particles, each representing an estimated motion state of the vehicle, superimposing the motion of the generated particles with noise, calculating a running average for each particle, weighting the superimposed particles according to a probability of agreement, and periodically resampling the particles according to the weights. In this case, a particle denotes a hypothesis of a possible localization.

[0015] In one embodiment, the sensor data includes objects located in the vehicle's vicinity. This allows the generated particles to be compared with reality based on the sensor data. The objects serve as a reference point.

[0016] In one embodiment, the state of motion of the vehicle includes the location, speed, or yaw rate of the vehicle.

[0017] In one embodiment, the noise has a normal distribution.

[0018] In one embodiment, the probability of a match is determined by comparing the particles with the previously recorded sensor data.

[0019] In one embodiment, periodic re-sampling takes place, so that particles with a higher weighting are more likely to be selected.

[0020] In one embodiment, the noise has a distribution with a mean of 0.

[0021] In one embodiment, exponential smoothing of the running average is used. This has the effect of improving the applicability of the method according to the invention to time-varying calibration errors.

[0022] In one embodiment, the particles are propagated according to information about a measured vehicle condition.

[0023] According to the invention, the running mean is compared with a threshold value; and if the running mean is greater than the threshold value, the miscalibration is counteracted.

[0024] In one embodiment, a vehicle has devices that perform the previously described procedure. [Brief description of the illustrations]

[0025] The illustrations explain the concept of the invention, wherein: Fig. 1 shows an exemplary situation according to an embodiment of the present invention, and Fig. 2 shows a flowchart according to the present invention. [Detailed description of the illustrations]

[0026] Fig. Figure 1 shows an exemplary situation in which the present invention can be used. A road section 100 is shown, on which a vehicle 110 is located; the figure is not to scale. The vehicle can be a passenger car, truck, bus, motorcycle, or even an e-bike; it is clear that the present invention can be implemented in any type of vehicle.

[0027] Even if in Fig. Figure 1 shows a straight section of road 100, it is clear to the person skilled in the art that the present invention can also be used in curves or other road sections such as intersections or junctions.

[0028] To estimate the driving state of vehicle 110, the algorithm according to the invention generates hypotheses that indicate the vehicle state. The generated hypotheses are also referred to as particles. A Monte Carlo algorithm can be used to generate these particles. Monte Carlo techniques are known, therefore they will not be described in detail within the scope of this application. In general, Monte Carlo methods use a random number generator; within the scope of the inventive concept, random vehicle states, the aforementioned particles, are generated using this generator.

[0029] In the present case, the position of vehicle 110 is known or its position can be initialized, since vehicle 110 has a GNSS module, which enables satellite-based localization.

[0030] Furthermore, vehicle 110 may have a map, in particular a high-resolution map, which provides information about the surroundings of vehicle 110.

[0031] The vehicle 110 can also derive information about its own movement from odometry or steering wheel movements and use this information to determine its own movement. Such information may contain miscalibrations or may change during operation, for example, due to temperature-dependent tire pressure.

[0032] Based on the satellite-based position data, a multitude of particle hypotheses are generated in the vicinity of the determined position. These numerous hypotheses are in Fig. 1 indicated by elements 131-136. Each of these elements 131-136 represents such a particle; each particle possesses properties such as location, orientation, and / or velocity. The location is given by the dot, the velocity is indicated by the length of the arrow connected to the dot, and the orientation of the arrow indicates the orientation of the vehicle.

[0033] As in Fig. As can be seen in Figure 1, two particles 131 and 136 are oriented upwards, two particles 133 and 134 are oriented downwards, particle 132 points to the right, and particle 135 points to the left. The erratic distribution of the elements and their orientation and arrow length is intended to illustrate the random selection of a set of hypothesized particles; this corresponds to the basic concept of Monte Carlo simulation.

[0034] The location of the particle can be aligned with the determined position of vehicle 110; within the framework of the Monte Carlo method used, the alignment with the location of vehicle 110 can refer to the mean value of the locations of all generated particles.

[0035] In Fig. Figure 1 shows a traffic sign 150. The vehicle is equipped with a sensor, for example a camera, which can detect a reference point such as the traffic sign 150. As will become clear below, the traffic sign 150 represents a reference point with which the previously generated particles can be checked for their correspondence with reality. Instead of a traffic sign, any reference point can be used within the scope of the present invention. For example, prominent fixed points such as traffic lights, intersections, houses, bridges, road markings, or trees can be used in a method according to the invention. The exact position of such a reference point, such as the traffic sign 150, is assumed to be known and can, for example, be stored in available map data.

[0036] In the next step, based on the sensor measurement and for each of the estimated particles, the position of the reference point, here the traffic sign 150, would have in the coordinate system that applies to the respective particle under consideration is determined.

[0037] The sensor measurement, for example using a camera, provides the information that the reference point, in this case the traffic sign 150, is located near the vehicle, and also provides the relative position of the reference point, in relation to the position of the sensor.

[0038] In the Fig. In the situation shown, traffic sign 150 is located diagonally to the right and in front of the vehicle. Based on sensor information, the system determines the position of traffic sign 150 relative to the vehicle. Expressed in vehicle coordinates, this means that the traffic sign is located a certain distance A to the right of the vehicle and a certain distance B in front of the vehicle.

[0039] Next, for each of the estimated particles, the position of the traffic sign is calculated, assuming the particle corresponds to the actual position and orientation of the vehicle. For example, considering particle 135, whose arrow points to the left, it is indicated that the traffic sign should be located diagonally to the right in front of this particle. The hypothetical position of the traffic sign is indicated by an 'X' positioned diagonally in front of the particle. Fig. In Figure 1, this is illustrated by the hypotenuse of the right-angled triangle located in front of the tip of the particle's arrow, indicating that the assumed position of the traffic sign is a distance A to the right and a distance B in front of the vehicle. However, in this case, the assumed position of the traffic sign is far from the actual sign 150, namely in the grassy verge to the left of the roadway, at the assumed position X. Considering particle 131, the assumed position X of the traffic sign is quite close to the actual position of traffic sign 150. Relative to the particle, the assumed position is again shifted a distance A to the right and a distance B forward, and is indicated by a cross. Here, it can be seen that the assumed and actual positions are relatively close. Therefore, within this highly simplified situation, this particle represents the best estimate.

[0040] The distance of the cross 'X' of each particle from the actual position of traffic sign 150 is a measure of how well the respective particle corresponds to the real situation. Thus, the distance is a clear measure of the probability of agreement, indicating how well the respective particle reflects reality.

[0041] To improve the localization quality, a motion model can be applied to the particles according to an embodiment not covered by the independent claims. The vehicle 110, which is to be localized, moves at a certain speed on the road 100. The localization method according to the invention requires a certain amount of time to execute its steps, so that after completion of a run of the method, the vehicle will no longer be in the same position as when the execution of the method began. Thus, a speed-dependent error can creep in; to minimize this error, a motion model is used that models the movement of the vehicle 110.In the simplest case, this motion model can include the consideration of a linear change in position, but it can also take into account that the vehicle is driving around a curve, accelerating or braking; this can be represented by appropriate modeling.

[0042] A further improvement can be achieved by superimposing noise onto the motion of the generated particles. This noise preferably has a mean of zero; this can be achieved, for example, by using a Gaussian distribution. Based on the noisy motion of the particle, a moving average is calculated, and if the noise has a mean of zero, the moving average should be close to zero. If the moving average is not close to zero, this indicates a miscalibration. Before concluding that a miscalibration is the cause, it may be useful to perform a statistical significance test to determine whether the deviation from zero is merely due to chance. So far, it has been determined that the particle exhibits a miscalibration.The probability of the particle's presence, and thus the probability of miscalibration, can be determined by comparing the environmental sensor information with the map, assuming the position hypothesis, or by comparing the position hypothesis with the GNSS position. When determining the presence of a miscalibration, and depending on the probability associated with the particle, it can be concluded that a miscalibration exists only if the particle has a certain probability.

[0043] For the purposes of the considerations described, it can be assumed that the sensors and map used do not have a substantial offset that accumulates over time. This can generally be assumed for maps generated based on GNSS data, and it also applies to GNSS sensors. Miscalibration of cameras or radar sensors only leads to a static offset, not to a time-accumulated offset like the one that occurs with miscalibrated ego movement, which is observed, for example, in dead reckoning in navigation systems.

[0044] When generating (re-sampling) a new set of particles, the knowledge gained from the previously determined match probabilities can be incorporated. The Monte Carlo method used can be based on different distributions of random values. The simplest approach is to assume a uniform distribution within certain limits, or a cumulative distribution function such as a Gaussian distribution. Knowing the match probabilities allows the probability distribution used in the Monte Carlo simulation to be adjusted so that particles with a higher degree of resemblance to reality are preferentially generated. This can be achieved, for example, by applying a weight to each particle, with the weight being higher the closer the particle is to reality.The exact design of the weighting function can be adapted to the respective application; generally, a better match results in a higher weighting, and a worse match results in a lower weighting.

[0045] Fig. Figure 2 shows a flowchart of the steps taken within the scope of the present invention. The method described here corresponds to the method already described in connection with Fig. Section 1 has already been explained, therefore a repetition is avoided here; instead, reference is made to the explanations regarding... Fig. 1 referred.

[0046] In step S210, one or more particles, each representing an estimated state of motion for the vehicle, are generated. As previously described, the particles are generated randomly using the Monte Carlo simulation technique employed here. Each generated particle can correspond to an estimated state of motion, encompassing the vehicle's position, velocity, and / or yaw rate. The position can be specified in two or three dimensions; usually, specifying the position in two dimensions is sufficient, as the vehicle's ground-bound nature, combined with a map that may also include topographic information, inevitably determines its height. Limiting the simulation to two position coordinates or two velocity coordinates significantly reduces the computation time and memory requirements of the Monte Carlo technique used here.

[0047] In step S220, the motion of the generated particles is superimposed with noise. This means that a purely random component is added to the randomly determined particle values. A moving average is calculated for each particle. As mentioned earlier, this can be used to detect calibration errors.

[0048] In step S230, the particles superimposed with the noise are weighted according to a probability of agreement. For example, it can be determined whether the previously generated particles correspond to reality more or less well. If a particle only slightly corresponds to reality, it is assigned a low weight; if a particle corresponds well to reality, it receives a higher weight.

[0049] In step S240, particles are generated again, this time according to the previously determined weights. A particle with a higher weight has a greater probability of being generated again, while a particle with a lower weight has a very low chance of reappearing.

[0050] By taking the weights into account, the probability distribution used in the previously described Monte Carlo simulation is adapted to reality, so that, based on the changed probability distribution, the result of step S210, which is performed again during re-sampling, will be closer to reality.

[0051] Since step S240 is essentially the same as step S210, with the difference that step S240 uses a modified distribution function for particle generation, this is a periodically executable concept that includes information feedback.

[0052] In this way, it is possible to keep the number of randomly generated particles lower during re-sampling, thus saving computing time and memory.

[0053] Although the inventive method has been described in the context of a traffic situation on a road, the method can be used, among other things, in (partially) automated driving, in driver assistance systems, in automated parking, in so-called “Automated Guided Vehicles” and also in robots, for example in logistics.

[0054] Finally, it should be noted that terms such as "comprising," "encompassing," etc., do not exclude other elements or steps, and that terms such as "a" or "an" do not exclude a plurality. Reference numerals in claims are not to be interpreted as limitations.

Claims

[1] Calibration procedure for a vehicle that has sensor data, the procedure comprising: Generating (S210) several particles, each indicating an estimated motion state of the vehicle, Superimposing (S220) the motion of the generated particles with noise, whereby a running average is formed for each particle, Weights (S230) of the superimposed particles according to a probability of agreement, Periodic re-sampling (S240) of the particles according to their weights, Comparing the current mean with a threshold; and if the current mean is greater than the threshold: Counteracting miscalibration. [2] Method according to claim 1, wherein the sensor data comprise objects located in the vicinity of the vehicle. [3] Method according to any of the preceding claims, wherein the state of motion of the vehicle comprises at least one of a location, a velocity and a yaw rate. [4] Method according to any of the preceding claims, wherein the noise corresponds to a normal distribution. [5] Method according to one of the preceding claims, wherein the probability of agreement is determined by comparing the particles with the sensor data. [6] Method according to one of the preceding claims, wherein the periodic re-sampling of the particles is carried out such that particles with a higher weight are selected with a greater probability. [7] Method according to any of the preceding claims, wherein the noise has a distribution with a mean of 0. [8] Method according to any of the preceding claims, wherein exponential smoothing of the running mean is used. [9] Method according to any one of the preceding claims, further comprising: Propagating the particles according to information about a measured vehicle condition. [10] Vehicle comprising equipment configured to perform a method according to any one of claims 1 to 9.

Images