Increasing a certainty in a sensor fusion particle filter using fixed lasers without LiDAR properties

Abstract

The invention is concerned with a method for confirming at least one property of an object 29 at a given spatial position (X,Y,Z) in an environment 19 of a vehicle 10, the method comprising: generating property data 27 describing at least one property of the object using a sensor 30 of a vehicle while the object is within a detection range 31 of the sensor; tracking a relative motion 36 of the object with regard to the vehicle and associating a spatial position, as determined by the tracked relative motion, with the property data; obtaining camera 45 images of the environment outside the detection range from at least one camera of the vehicle; detecting at least one laser light patch 49, each associated with a respective spatial region (X',Y',Z') where the laser light patch is located in the environment according to the images, and associated with the object passing through a laser beam 44; and signalling a confirmation value if the respective spatial region matches the spatial position associated with the property data, thus confirming or increasing a certainty that the property data are effective at that spatial position.

Description

The invention is concerned with object detection by using at least one sensor of a vehicle for generating property data describing at least one property (e.g. radar radiation reflectivity and / or visible features) as they are observed at an object in the environment of the vehicle. More particular, the invention is concerned with tracking the object after it has left the detection range of the at least one sensor such that a certainty can be increased regarding the question where the at least one property is still effective in the environment outside the detection range. This confirmation of the presence of an object at an estimated spatial position is made possible without the use of a LiDAR, instead a fixed laser device is sufficient. A vehicle, like a motor vehicle, in particular a passenger vehicle or a truck, may surveil or observe its environment for the purpose of detecting objects that need to be considered while moving the vehicle through the environment. Such an object may be a stationary object, like traffic infrastructure (e.g. traffic signs, road marks, bridges, road lanes), or dynamic or moving objects, like e.g. other traffic participants (e.g. vehicles, cyclist, pedestrians) and / or moving obstacles (e.g. debris moved by the wind and / or toys from children). Depending on the type of object, the vehicle may either use such an object for adjusting its own position (like in the case of road marks and / or traffic signs) and / or the detection result may be used for collision avoidance (e.g. concerning other traffic participants). For detecting an object, the vehicle may comprise at least one sensor that has a detection range (which may be a predefined detection range) that is directed towards or into the surrounding environment. From measurement data of the at least one sensor, a processor circuit of the vehicle may derive or accumulate property data of at least one such object in the environment. For example, a radar sensor may provide radar measurement data that describe reflection events of radar radiation at a certain direction and distance from the vehicle. As property data, the processor circuit may therefore arrive, that at that point in the environment an object with a certain radar radiation reflectivity exists. Together with such property data describing properties of an object, the position of that object may be measured and tracked. In other words, the property data describing the at least one property (e.g., radar radiation reflectivity or presence of a visible feature) may be associated with position data describing where the at least one property of the object is currently positioned or effective. A vehicle might not have as many sensors as might be necessary for covering the whole surrounding of the vehicle, i.e. a 360 degree coverage by detection ranges of sensors. In particular, the at least one sensor may have its detection range directed towards a front area (forward driving direction) of the vehicle and in some cases also to the rear area (backward driving direction) of the vehicle. Only while an object is in a relative position to the vehicle such that it is within the detection range of the at least one sensor, the property data for determining or describing the at least one property of the object may be determined or measured or accumulated. Due to a relative movement or motion of the vehicle with regard to the object (may it be due to a self-motion (ego motion) of the vehicle and / or a motion of the object (dynamic object)), the object may leave or move outside the detection range. From then on, for updating the information on the spatial position where the property data are currently effective (e.g., currently accurate), a motion tracking may be used that may track the relative motion of the object with regard to the vehicle which allows to update said position data associated with the property data. The motion tracking may be based on odometric measurements regarding the ego motion of the vehicle itself and / or on modelling the motion of the object through the environment (e.g. based on a dynamic model). As the object is outside the detection range, the assumption might have to be made that the object continues its own motion through the environment with the same dynamic characteristics (velocity, direction of motion and / or acceleration) as was last observed while the object was still in the detection range of the at least one sensor. In other words, a change of the dynamic characteristics of the object might lead to an estimation error in the motion tracking. Another source of error might be the odometric measurement itself such that the position of the vehicle in the environment might be estimated falsely such that the relative position of the object with regard to the vehicle might also be misjudged. Thus, the position data associated with the property data might not match the true spatial position where the object’s properties are currently in effect. Prior art is known from the publication US11520024B2 describing a LiDAR-based tracking system. The publication US10753736B2 describes a projected pattern of laser dots detected by respective camera devices in a physical environment, and a stereoscopic two-dimensional (2D) object pair based on determining 2D positions for each of the laser dots detected in the first and second images. This solution requires stereoscopic image analysis. It is an objective of the present invention to verify or confirm that the property data describing at least one property of an object are still effective or valid regarding their estimated spatial position once the object has left the detection range of the at least one sensor of the vehicle due to relative motion of the object with regard to the vehicle. This objective is achieved by a method, a processor circuit and a motor vehicle as described in the independent claims. Further beneficial developments of the invention are described by the dependent claims, the following description and the drawings. As one solution the invention provides a method for confirming at least one property of an object at a given spatial position in the environment of a vehicle. For the following description, it may be assumed that the “object” may be a stationary object or a dynamic object like has been described above. Such an object may be detected by the processor circuit of the vehicle using an object detection and / or object classification (as is available from the prior art). Alternatively, the “object” may be a collection of property data that may have been observed in combination in a predefined observation volume, for example a cube of edge length in the range of e.g. one millimeter to 50 centimeters. In other words, the “object” can be a collection of property data as available in a pre-detection stage and / or preclassification stage of data processing of the processor circuit. In that sense, the “object” can be regarded as a combination of property data and a position data where the property data are currently located. The “spatial position” can be denoted as a 3D coordinate, for example. In order to accumulate or derive the property data, the method may comprise the step of generating, by the processor circuit, the property data describing the at least one property of the object using one predefined sensor or several predefined sensors of the vehicle, while the object is within a respective detection range of the at least one predefined sensor. As has been described above, the “property” may be, for example, radar density and / or relative speed and / or acceleration (as can be measured by a radar sensor), LiDAR reflectivity (as can be measured by a LiDAR sensor) just to name examples. It is to be noted that that in preferred embodiments a front LiDAR is used for the measurement of the property data of the object. The detection range of the at least one predefined sensor may be a common detection range, i.e. in the case of several predefined sensors their detection ranges overlap. As one example, the respective detection range of the at least one predefined sensor may be directed towards the above-mentioned front area, i.e. it may point or be directed towards the forward driving direction. Further, the method comprises tracking, by the processor circuit, a relative motion of the object with regard to the vehicle and associating a spatial position as determined by the tracked relative motion with the property data such that the property data associated with the spatial position. The tracking or tracking procedure may be based on a solution taken from the prior art. When the object leaves the detection range, the invention provides a possibility to at least confirm or verify that the property data are still effective or positioned at the spatial position as estimated or provided by the tracking. There is a certainty concerning the spatial position of that object that is being tracked. This certainty is decreasing over time. It is therefore of advantage to increase the certainty. To this end, the method comprises projecting or emitting, by at least one laser device of the vehicle, at least one laser beam into the environment, that is into the environment outside the respective detection range of the at least one sensor. Thus, as a result, when due to the above-mentioned relative motion, the object moves out of the detection range and passes through the at least one laser beam, at least one laser light patch or laser dot is generated on the surface of object. The laser light may comprise visible light and / or infrared light, just to name examples. In the above example where each detection range covers the described front area, more laser beams may be projected to the left and right of the vehicle, i.e. into lateral areas next to the vehicle. In particular, a laser device with a fixed laser beam and without LiDAR capabilities can be used. Additionally, the method comprises obtaining, by the processor circuit, camera images of the environment outside the detection range from at least one camera of the vehicle. In other words, the environment outside the detection range is observed or monitored using at least one camera of the vehicle. In particular those regions of the environment where the at least one laser beam passes through is filmed or covered by the at least one camera. Thus, if the object is still at the estimated spatial position, consequently there should also be a laser light patch observable at this spatial position (as long as the object is not transparent for the laser light or 100% reflective without any light scattering). For confirming the estimated spatial position, the method comprises detecting, by the processor circuit, the at least one laser light patch in the camera images. In other words, the camera images or their image data, are analyzed for determining whether the image data contains an optical image of the laser light patch. To this end, in the case of infrared light, an infrared camera may be used and the detecting of the laser light patch may comprise searching for image data or pixels (pixel elements), that exhibit or describe a brightness level or intensity larger than a predefined threshold. When visible laser light is used, the analysis may comprise detecting image data or pixels with a brightness level or intensity larger than a predefined threshold for the light color of the laser light. From the position of a detected laser light patch in a camera image, a corresponding spatial region may be derived that describes where the laser light patch must be positioned in the environment. In other words, each detected laser light patch is associated with a respective spatial region where the laser light patch is located in the environment according to the camera images. Details regarding advantageous geometric calculations for deriving the spatial regions are described below. The method also comprises signaling (e.g., outputting), by the processor circuit, a confirmation value or confirmation signal, if this respective spatial region associated with the at least one laser light patch matches the spatial position associated with the property data. For example, only if their spatial or Euclidean distance is smaller than a predefined threshold (e.g. in the range of 1mm to 1m), the confirmation may be signaled. Time stamps may be provided for both the estimate of the spatial position of the object and the image showing the laser light patch. The comparison may include comparing the time stamps. The confirmation value may only be generated if the time stamps differ less than a predefined threshold that may be in a range from Os to 200ms. Thus, the method provides a confirmation that the property data are still effective at that spatial position. The method therefore is based on the insight that when property data can be derived from an object based on measurement data of at least one sensor, that object must be dense enough that laser light will also provide a laser light patch when a laser beam hits that same object. On the other hand, if there is a discrepancy between the estimated relative motion of the object and the true relative motion of the object with regard to the vehicle, the laser light patch will be positioned on the object but at a different (i.e. the true) spatial region, whereas the property data will be associated with the (erroneously) estimated spatial position based on the tracking. In such a case, the spatial position may either be corrected to be equal to the spatial region of the detected laser light patch (if a predefined similarity criterion is met, e.g. distance smaller than a threshold, that can be in a range of 1 mm to 2m), or additionally or alternatively a degree of certainty regarding the validity of the property data and / or their spatial position may be decreased. This is provided by a beneficial further development described below. The confirmation value may be, for example, a counter value or “certainty bonus” confirming that a laser beam has hit a reflective surface in a spatial region where also property data are assumed according to their spatial position. For each laser patch, such a confirmation value may be added or provided. In other words, when the object passes several light beams, for each resulting laser light patch the associated confirmation value may be noted. The invention also comprises further beneficial developments that provide additional technical effects. One further development uses fixed lasers. This development comprises that the at least one camera and the at least one laser beam are kept at a fixed relative orientation with regard to the vehicle and the respective spatial region where the laser light patch is located in the environment is derived from a predefined relative geometric arrangement of the at least one laser device and the at least one camera. In other words, the setup comprising the at least one laser device and the at least one camera is static with regard to the vehicle. This allows to derive the spatial region based on constant or non-changing geometric data describing the geometric arrangement and from an image position of the detected laser light patch. A suitable geometric equation can be taken from the prior art. For a more precise estimate of the spatial region, a respective current value for pitch angle and / or roll angle can also be considered in the calculation. One particular benefit is obtained, when the at least one laser device projects the laser beam at an angle unequal to 0 with regard to the horizontal plane (measured while pitch angle and roll angle are zero, e.g. during standstill). In other words, the laser beam has a declining or ascending direction starting from the laser device. The method may then further comprise: estimating, by the processor circuit, the distance of the respective laser light patch from the vehicle (in particular from the laser device) as a geometric function comprising as input parameters the value of the angle and a vertical position where the respective laser light patch is detected in the respective camera image. In the case of an ascending laser light beam, the higher the laser light patch is positioned in the camera image, the further away the laser light patch is. Likewise, for a descending laser beam, the lower the laser light patch is positioned in the camera image, the further the laser light patch is positioned away from the vehicle. This allows to derive or estimate the distance of the laser light patch from the vehicle using a single laser light beam, in particular without time-of-flight measurement. In particular, no LiDAR measurement is necessary for localizing the laser light patch in the environment. The geometric function may comprise a geometric equation and / or a look-up-table. Consequently, a further benefit may be obtained, when the camera images are obtained as monoscopic images from a respective monoscopic image sensor of the at least one camera. In other words, it is sufficient to detect the laser light patch in one single monoscopic image, i.e. an image without 3D or z-coordinate or depth information. This makes detecting the laser light patch cheap with regard to equipment costs. However, other embodiments of the invention may use stereoscopic images. Generally, any kind of spatial localization for localizing a laser light patch based on a single camera image or multiple camera images can be used for implementing the method. Generating and signaling said confirmation value has proven beneficial for controlling or parameterizing a function for controlling the vehicle. Accordingly, one further development is given, if the method further comprises using the property data and / or data derived from those property data (for example a detection result and / or a classification result) in at least one controlling application for controlling the vehicle, wherein a respective degree of influence of the property data and / or of the data derived from the property data on that controlling application is set as a function of their confirmation value. The lower the confirmation value or the less confirmation values have been signaled due to fewer laser beams hitting the object, the lower the influence of the corresponding property data or the data derived therefrom on the controlling application. Likewise, the higher the confirmation value, the higher the influence can be. This mirrors the idea that if the object cannot be found or detected at the spatial position where it should be according to the tracking of the motion, the object must have changed its trajectory compared to the estimated trajectory from the tracking, such that the controlling application may not rely or may rely less on the property data. For this purpose, the confirmation value may be transformed into a weight or weighing value that may be applied to a calculation result that is based on the property data or the data derived therefrom. For example, a probability value chosen from the interval 0 percent to 100 percent or an equivalent value thereof, e.g. 0 to 1, may be derived from the confirmation value in order to provide such a weighing value. A further benefit is obtained, when additionally to the confirmation value, an absolute confidence or certainty regarding the validity of the position data associated with the property data is made available, for example for the above-described controlling application and the influence thereon. In this regard, one further development comprises that generating the property data comprises: iteratively updating the property data by the processor circuit, by accumulating sensor data from the at least one sensor, and associating a certainty value with the property data, wherein the certainty value is increased with each iteration and wherein the certainty value indicates a progress of the accumulation and / or indicates a variance of the sensor data. For example, the certainty value may comprise the number of iterations during which sensor data could successfully be integrated or accumulated into the property data. A value for such a certainty value can be in the range of, e.g., 0 to 5000. The variance of the sensor data can be expressed, e.g., as the statistical variance of the underlying sensor data. Thus, while the object is within the described detection range, iterations of measurement cycles at a rate of e.g., 1 Hz to 100 Hz, may be performed and with each measurement cycle, new measurement data may be acquired from the object. When the object leaves the detection range, a further accumulation of sensor data must be stopped, thus the final certainty value provides a notion of certainty with regard to the property data. Note that associating position data with the property data while the object is within the respective detection range can be implemented based on a localization method from the prior art, e.g. a camera and / or LiDAR based localization. Said certainty value can then be combined with the at least one confirmation value (each laser light patch may provide one confirmation value) to indicate an overall current certainty with regard to the property data. In this context, one beneficial further development comprises that when the object has moved out of the detection range, the updating of the property data for the object is less rich (i.e. further accumulation of further sensor data is stopped or at least reduced, if the object leaves the detection range of one sensor, e.g. LiDAR, and is still in the detection range of another sensor, e.g. RADAR) and the certainty value is (continuously or stepwise) decreased over time. This decreasing of the certainty value indicates the “aging” of the property data as no further measurements or updates are available. For example, the decreasing may be implemented as an iterative performance of subtracting a predefined aging value, for example in the range of 10 to 200, from the initial certainty value when the object has left the at least one or each detection range. Thus, a decreasing certainty value is given, but additionally for each laser light patch that matches the spatial position associated with the property data, an increase of a confidence value or an additional confidence value is provided in the form of the confirmation value. For evaluating the current overall reliability or certainty regarding the property data, the method may comprise calculating, by the processor circuit, a combination of the certainty value and the respective confirmation value, wherein the combination indicates or has the effect of a delay and / or interruption and / or reversal of the decrease in overall certainty of the property data. For example, for each laser light patch detected at the spatial position, a pre-defined confirmation value may be added or provided such that the decrease in certainty value is compensated in the described way in the combination. The combination can be, for example, the sum of this current certainty value and each confirmation value associated with the property data. This indicates a “refresh” of the certainty whenever a laser light patch is detected at the spatial position. Aging (decrease over time) may also applied to each confirmation value. In preferred embodiments, different sensors, in particular LiDAR, RADAR and / or Camera image analysis, are all contributing to the property data of the object (e.g. a volume cube). As has been described already, the method may of course be implemented for several laser beams, i.e. the method may comprise providing a separate confirmation value for each of the at least one laser light beam and storing by the processor circuit the respective confirmation value separately from the certainty value such that the certainty value keeps decreasing over time and the overall reliability is calculated using a sum of the certainty value and each confirmation value. This allows for evaluating two different laser beams or more than two different laser beams. Several laser beams may be provided using several laser devices and / or at least one sparkle laser device, i.e. a laser device may be designed to project one or several laser beams at the same time. Each laser beam may be directed into a different direction. The respective laser beam may be un-modulated or modulated with regard to density and / or color. The phase information of the laser may be unknown, no time-of-flight analysis may be performed for the method. As has been described before, the “object” can be a static object or a dynamic object, as may have been detected or classified (recognized) by a machine learning procedure and / or by a recognition procedure that deterministically and / or analytically analyzes the object properties (for example, a body with properties of water and height of 1.7 meters can be classified as a human body). The latter analytic object-detection process may be implemented as an expert system. However, in one further development, the “object” can be a volume element, that may be part of a larger body, for example a volume element of a traffic participant and / or a traffic infrastructure element. Such a volume element can be of the mentioned size, e. g. a cube of edge length in the range from 2 centimeters to 50 centimeters, just to name examples. In a memory of the processor circuit, the object can be modelled or represented by a data structure that represents that the spatial volume element together with adaptable or with absolute, constant geo-spatial coordinates of the volume element in the environment. This describes two cases: Adaptable coordinates mean that the data structure may also store the current coordinates, either relative to the vehicle or absolute geo-spatial coordinates, that are changed whenever the tracking indicates a change in spatial position. In contrast to this, absolute coordinates can be used, if the volume element represents a spatially fixed volume in space and when the tracking of the motion indicates a movement of the object into that volume element, the property data from the data structure are copied into the part of the memory, that represents that volume element. Such a technique can be adapted from, for example, the patent application US 18 / 069,686 filed with the US patent and trademark office as application “hybrid particle filter application”. The description of static and dynamic objects using geo-spatial coordinates of absolute, spatially fixed volume elements as described in that application is incorporated in this present application by reference. As the described volume element may be smaller than an actual traffic participant and / or a traffic infrastructure element, a further development is obtained, when, by the processor circuit, the respective property data of several such spatial volume elements (i.e. several “objects”) are provided to a machine-learning-based object-classifier process of the processor circuit or to an analytic object-detection process as described above, wherein the respective process performs an object detection and / or classification (object recognition) for recognizing a predefined traffic infrastructure element (e.g. a guard rail or a traffic sign) and / or a dynamic traffic participant (e.g. another vehicle) in the environment based on the property data from the several spatial volume elements. In other words, the data structure for the volume element can be seen as an early sensor fusion data structure that collects property data of different properties of the object and this is also done at a pre-classification stage, i.e. the “object” is not classified or recognized as such (i.e. the optic type is unknown) for the steps of comparing the spatial region of laser light patches with the spatial position of the object. This provides the advantage that no complex object detection or object classification needs to be performed for confirming the property data at the spatial position. The confirmation values may also be used for assessing the quality of the recognition result in that the method may further comprise, signaling, by the processor circuit, the recognized traffic infrastructure element and / or traffic participant together with its position in the environment (as derived from the spatial positions associated with the volume elements) to a driver assistance functionality that plans and / or executes a driving trajectory that leads the vehicle through the environment. In other words, the detection or recognition results may be used for an automated or autonomous driving functionality in the vehicle. Additionally or alternatively, the recognized or classified object may be mapped or noted to a mapping functionality that generates and / or updates a dynamic map of the environment. The recognition result is thus entered into that dynamic map. This map may be used as a basis for planning the described driving trajectory. The influence of the recognition result on the planning of the driving trajectory can be weighed or influenced in the described way based on the confirmation value or the several confirmation values. As has already been described, the property data may comprise property values of different properties of the object such that the property data may implement a sensor fusion. The respective property value may be calculated as a mean value of sensor data obtained from successive measurements with a corresponding sensor for that property, e.g. a radar or a LiDAR or an ultrasonic sensor and / or an infrared camera and / or a color image camera, just to name examples. Additionally or alternatively to calculating the mean value, a particle filtering may be used where the sensor data is used as a filter input for a particle filter process of the processor circuit. Particle filtering has proven particularly robust for processing sensor data in the described situation. The tracking of the relative motion of the object with regard to the vehicle may comprise an odometric measurement of the movement of the vehicle itself through the environment. This odometric measurements may comprise readings from a data bus of the vehicle, e.g. a CAN bus (controller area network), and / or an inertia measurement and / or operating a model for transfering the odometric reading and / or the inertia values to motion values and / or a camerabased odometry. If the object is a dynamic object, the motion vector of the object and / or acceleration values for different degrees of freedom may be measured while the object is in the detection range of the at least one sensor, and the motion of the object may be extrapolated into the environment outside the detection range. A solution to the above-stated objective is also given by a processor circuit comprising instructions that when executed by the processor circuit cause the processor circuit to perform the following method steps confirming a spatial position of an object in an environment of a vehicle: • generating, by the processor circuit, property data describing at least one property of the object using at least one predefined sensor of the vehicle, while the object is within a respective detection range of the at least one predefined sensor, and • tracking, by the processor circuit, a relative motion of the object with regard to the vehicle and associating with the property data a spatial position of the object according to the tracked relative motion, and • obtaining, by the processor circuit, camera images of the environment outside the detection range from at least one camera of the vehicle and • detecting, by the processor circuit, at least one laser light patch in the camera images, wherein each detected laser light patch is associated with a respective spatial region where the laser light patch is located in the environment according to the camera images, and • signaling, by the processor circuit, a confirmation value, if the respective spatial region associated with the at least one laser light patch matches the spatial position associated with the property data, thus confirming that the property data are still available at the spatial position. Such a processor circuit can be based on at least one microprocessor and / or at least one ASIC (application specific integrated circuit). The method step may be implemented as hardwired processing and / or as computer-readable instructions (e.g., instructions on a non-transitory computer-readable medium, such as memory storing the instructions) that when executed by the processor circuit cause the processor circuit to perform the method steps that need to be performed by the processor circuit according to the method described above. The instruction can be stored in a memory of the processor circuit that may be coupled to or integrated in the at least one microprocessor and / or the at least one ASIC. The processor circuit may be designed as a SoC (system on chip) or as a MSoC (Mulitiple Systems on Chip). A solution to the above-stated objective is also given by motor vehicle comprising the at least one sensor with a respective detection range directed into an environment of the vehicle, the at least one laser device, the at least one camera and the above-described processor circuit, wherein the vehicle is designed to perform a method according to any of the described method or its further developments. The invention also comprised the combination of the features of different further developments of the invention. Any situation that might be encountered while performing the method might be handled as an error situation where the method might signal an error and / or a reset of the processor circuit might be performed. In the following, implementation examples of the invention are described. The figures show: Fig. 1 a schematic illustration of a motor vehicle according to the invention; Fig. 2 a schematic illustration of the vehicle in an environment; Fig. 3 a diagram illustrating an embodiment of the inventive method; and Fig. 4 a sketch for illustrating a procedure for estimating a distance of an object with regard to the vehicle. Fig. 1 shows a bird view of a vehicle 10 that can be, for example, a passenger vehicle or a truck or in general a motor vehicle. In the vehicle 10 a processor circuit 11 may be provided that may perform an automated driving function 12, for example a driving function of an automation level in the range of 2 to 5 according to the standard SAE J3016 of the norming organization SAE international (SAE - society of automotive engineers). Additionally or alternatively a driver assistance functionality 13 may be provided (e.g. lane assist). The automated driving function 12 and / or the assistance functionality 13 may generate commands 14 for controlling at least one actuator 15 of the vehicle, for example a brake and / or a steering and / or an acceleration for performing or executing a driving trajectory 16 that may be computed by the respective function 12, 13. The calculation of the driving trajectory 16 may be based on map data 17 of a digital dynamic map 18 which may be a digital model of an environment 19 of vehicle 10, i.e. the surroundings of the vehicle 10 may be modeled in the map 18. For entering or noting stationary traffic infrastructure elements 20, like for example, the road 21 the vehicle 10 is driving on and / or traffic signs 22 and / or for mapping or noting dynamic or moving traffic participants 23, like another vehicle 24 and / or other moving elements or obstacles, the map data 17 of the map 18 may be provided or updated with recognition data 25 describing such infrastructure elements 20 and / or dynamic traffic participants 23. The recognition data 25 may be generated by an object recognition 26 that may be based on an object detection (indicating a shape and / or position) and / or object classification (indicating an object type). The object recognition 26 may be provided with property data 27 as an input. The property data 27 may be properties of the different elements 20, 23 as they have been observed in the environment 19. The property data 27 may be associated with a respective single volume element 28 that can be seen as an “object” 29 in itself. In other words, before the object recognition 26, property data 27 of different single “objects” 29, like the volume elements 28 may be determined. These objects 29 are, of course, connected, if the belong to the same body, like a traffic sign, but may only become apparent at the later object detection and / or object recognition stage. For generating the property data 27, at least one sensor 30 may be provided and a respective detection range 31 of each of these sensors 30 may be directed towards a specific region or area of the environment 19, for example the front area 32 of vehicle 10. The vehicle 10 may be driving in a driving direction 33 through the environment 19. A speed of the vehicle can be in the range of 1 km / h to 350 km / h. Additionally or alternatively, the respective object 29 may be in self motion through that environment 19. While the object 29 is within the respective detection range 31 of the at least one sensor 30, the at least one sensor 30 may generate sensor data 34 depending on or describing the properties of the respective object 29. For example, one sensor 30 can be a radar, one sensor 30 can be a LiDAR, further examples are given above. Additionally, a relative motion 36 of the object 29 may be tracked. For example, using a radar as sensor 30, a relative velocity and / or acceleration of the object 29 with regard to the vehicle 10 may be determined. Additionally or alternatively, a video sequence showing object 29 may be used for determining the relative motion 36. For easier understanding, one single object 29 is regarded in the following. In Fig. 1, object 29 is shown at different points in time, in particular time T1 when object 29 is within the at least one detection range 31, and time T2, when object 29 is outside the respective detection range 31. When object 29 leaves the at least one detection range 31, the tracking of the relative motion 36 may be used to predict or estimate a respective current position X, Y, Z of object 29 in the environment 19 outside the detection range 31. As the property data 27 may not be updated any more at time T2, at least the validity or effectiveness of the property data 27 in the environment 19 with regard to the position X, Y, Z (where they should be effective) may be confirmed. The relative motion 36 may be estimated from odometric data 40 as they may be derived from a wheel rotation of wheels 41 of vehicle 10 and / or from camera images of a camera 42 allowing to derive the ego motion of vehicle 10 through environment 19. The motion of object 29 in the environment 19 may also be derived from or using camera 42 as it is known from the prior art. For confirming the correct spatial location of the property data 27 while object 29 is outside the respective detection range 31, vehicle 10 may comprise a laser device 43 for projecting at least one laser beam 44 into the environment 19 outside the detection range 31. Each laser device 43 may project (e.g., emit) one or several laser beams 44 into the environment 19. At least one camera 45 may provide image data 46 describing camera images of the environment 19 outside detection range 31 where the at least one laser beam passed through. When object 29 passes through one laser beam 44 (e.g., at least a portion of the object intersects with the laser beam 44), as is shown in Fig. 1 for time T2, a laser light spot or a laser light patch 49 is generated on the surface of the object 29 (e.g., appears on the portion of the object intersecting with the laser beam 44). the processor circuit 11 may process the image data 46 of the camera 45 together with geometric data 48 describing the geometric arrangement of laser device 43 and camera 45 in vehicle 10 such that from the image data 46 coordinates X', Y', Z of laser light patches 49 of the laser beam 44 on the object 29 may be derived. The camera detection range 50 of camera 45 may provide a camera image where the laser light patch 49 has a position that allows to derive the coordinates Xz, Yz, Z of laser light patch 49 in the environment 19. This is further illustrated in Fig. 3 and Fig. 4. The respective laser device 43 can be arranged at any angle with respect to the skin of the vehicle 10. Fig. 3 illustrates the situation of Fig. 1 (still from top view) in more detail with regard to object 29 at time T1 and T2. During time T1, the properties according to property data 27 may still be certain as measurements using the at least one sensor 30 for the property data 27 may be performed. Due to the relative motion 36, object 29 moves to the position X, Y, Z at time T2 which is outside the detection range 31 according to Fig. 3. There the laser beam 44 may hit object 29 causing the laser light patch 49 on the surface of object 29. This may be seen or described by the image data 46 describing a camera image 52 of the environment 19. Fig. 4 illustrates the situation from a front view further showing that with regard to a horizontal plane 53 the laser light beam 44 may be inclined i.e. not horizontal. In the camera image 52, the processor circuit may locate the laser light patch 49 on object 29 at coordinates u, v, wherein u is the horizontal position and v the vertical position of the laser light patch 49 in image 52. Comparing the coordinate of a virtual representation of the horizontal plane 53 in the image 52, the coordinate v compared to the horizontal plane 53 results in a difference dv which indicates the distance 57 of the laser light patch 49 with regard to laser device 43 as is illustrated in Fig. 4. The horizontal position u of laser light patch 49 in the image 52 and the geometric data 48 of the geometric arrangement of camera 45 and laser device 43 allow to derive the relative horizontal position of the laser light patch 49 with regard to vehicle 10. This may result in the estimated spatial region Xz, Y', Z of the laser light patch 49 in the environment 19. The coordinates X, Y, Z and Xz, Yz, Z may be, for example, relative coordinates in a coordinate system of the vehicle 10 and / or absolute geo-coordinates. Comparing the estimated coordinates X, Y, Z of the spatial position of object 29 and the estimated coordinates X', Yz, Z of the spatial region of the laser light patch 49 in a comparison 60 gives an indication or confirmation that the laser light patch 49 is on object 29. This can be signaled by a confirmation value 61. The comparison 60 can include a comparison of time stamps. A first time stamp can be generated by the tracking indicating that the coordinates X, Y, Z are valid for a certain time point T2. A second time stamp can be generated for the camera image 52 indicating the time when the image 52 was taken. In the case that several objects are tracked (e.g. several of the described cubes), the comparison of time stamps can be used for identifying the correct object. Overall, the property data 27 and additional memory for confirmation value 61 and the coordinates X, Y, Z from the tracking of the relative motion 36 may be stored in a data structure 62 describing object 29. While object 29 is within the at least one detection range 31, the property data 27 may be accumulated from the sensor data 34, resulting in the already-described confidence value 63. Once object 29 leaves the at least one detection range 31, and no update of the property data 27 is possible, a declining function 64 reducing the confidence value 63 over time t may be applied indicating the aging of the confidence into the property values according to property data 27. The confidence value 63 and each confirmation value 61 may be combined in a combination 65, e.g. a sum, to result in an overall certainty or reliability 66 regarding property data 27. The property data 27 together with their overall reliability value of certainty or reliability 66 may be provided as input for the object recognition 26. Therefore, while the confidence value 63 is continuously decreasing according to the decrease of declining function 64, reducing reliability 66, the restoration of reliability 66 may be obtained by each confirmation value 61. Fig. 2 illustrates how the environment 19 may be modeled by a plurality of objects 29 in the implementation as volume elements 28. For each such object 29 the described procedure may be performed such that the map 18 models the environment 19 based on such volume elements 28 with their property data 27. In other words, a memory M may store a plurality of data structures 62 as described. The object recognition may then interpret or infere the object type that they belong to. In particular, beneficial aspects of the invention are given by one or more of the following: Generating Measurement to increase confidence into a point cloud particle filter using fixed lasers and cameras may therefore be based on the following aspects. Aspect 1: There are cameras looking side way on the A and B pillars of the car. Aspect 2: Lasers are pointing on the side of car and creating dots on the side of the car or colliding with object into the space. Aspect 3: It is possible to use cameras and reconstruct the position of the dots made by the camera in 3D. Aspect 4: The additional lasers will allow to improve the certainty of the delta cube in the particle filter. Reference-signs vehicle processing circuit automated driving function driver assistance functionality commands actuator driving trajectory map data map environment infrastructure elements road traffic sign traffic participant motor vehicle recognition data object recognition property data volume element object sensor detection range area driving direction sensor data motion odometric data wheel camera laser device laser beam camera image data geometric data laser light patch detection range camera image horizontal plane distance comparison confirmation value data structure confidence value decreasing function combination overall reliability spatial position spatial region

Claims

1. Method for confirming at least one property of an object (29) at a given spatial position (X,Y,Z) in an environment (19) of a vehicle (10), the method comprising:• generating, by a processor circuit, property data (27) describing the at least one property of the object (29) using at least one predefined sensor (30) of the vehicle (10), while the object (29) is within a respective detection range (31) of the at least one predefined sensor (30), and• tracking, by the processor circuit, a relative motion (36) of the object (29) with regard to the vehicle (10) and associating with the property data (27) a spatial position (X,Y,Z) as determined by the tracked relative motion (36), and• projecting, by at least one laser device (43) of the vehicle (10), at least one laser beam (44) into the environment (19) outside the respective detection range (31), such that when due to the relative motion (36), the object (29) moves out of the detection range (31) and passes through the at least one laser beam (44), at least one laser light patch (49) is generated on the object (29), and• obtaining, by the processor circuit, camera (45) images of the environment (19) outside the detection range (31) from at least one camera (45) of the vehicle (10) and• detecting, by the processor circuit, the at least one laser light patch (49) in the camera (45) images, wherein each detected laser light patch (49) is associated with a respective spatial region (X',Y',Z') where the laser light patch is located in the environment (19) according to the camera (45) images, and• signaling, by the processor circuit, a confirmation value (61), if the respective spatial region (X',Y',Z') associated with the at least one laser light patch (49) matches the spatial position (X,Y,Z) associated with the property data (27), thus confirming or increasing a certainty that the property data (27) are still effective at that spatial position (X,Y,Z).

2. Method according to claim 1, wherein the at least one camera (45) and the at least one laser beam (44) are kept at a fixed relative orientation with regard to the vehicle (10) and the respective spatial region (X',Y',Z') where the laser light patch (49) is located in the environment (19) is derived from a predefined relative geometric arrangement of the at least one laser device (43) and the at least one camera (45).

3. Method according to any of the preceding claims, wherein the at least one laser device (43) projects the laser beam (44) at an angle unequal to zero with regard to a horizontal plane (53), and the method further comprises: estimating, by the processor circuit, thedistance (57) of the respective laser light patch (49) from the vehicle (10) as a geometric function comprising as input parameters the value of the angle and a vertical position where the respective laser light patch (49) is detected in the respective camera (45) image.

4. Method according to any of the preceding claims, wherein the camera (45) images are obtained as monoscopic images from a respective monoscopic image sensor (30) of the at least one camera (45) or the camera (45) images are obtained as stereoscopic images or any kind of spatial localization.

5. Method according to any of the preceding claims, wherein the method further comprises: using the property data (27) and / or data derived from the property data (27) in at least one controlling application for controlling the vehicle (10), wherein a respective degree of influence of the property data (27) and / or of the data derived from the property data (27) on the controlling application is set as a function of their confirmation value (61).

6. Method according to any of the preceding claims, wherein generating the property data (27) comprises: iteratively updating the property data (27), by the processor circuit, by accumulating sensor (30) data from the at least one sensor (30), and associating a certainty value with the property data (27), wherein the certainty value is increased with each iteration and wherein the certainty value indicates a progress of the accumulation and / or a variance of the sensor (30) data.

7. Method according to claim 6, wherein the method further comprises:• when the object (29) has moved out of the detection range (31) due to the relative motion (36), stopping updating the property data (27) and decreasing the certainty value over time; and• for evaluating a current overall reliability (66) of the property data (27), calculating, by the processor circuit, a combination (65) of the certainty value and the confirmation value (61), wherein the combination (65) indicates a delay and / or interruption and / or reversal of the decrease in certainty of the property data (27).

8. Method according to any of the preceding claims 6 or 7, further comprising: providing a separate confirmation value (61) for each of the at least one laser beam (44) and storing, by the processor circuit, the respective confirmation value (61) separately from the certainty value such that the certainty value keeps decreasing over time and the overall reliability (66) is calculated using a sum of the certainty value and each confirmation value (61).

9. Method according to any of the preceding claims, wherein, in a memory of the processor circuit, the object (29) is modelled by a data structure (62) that represents a spatial volume element (28) with adaptable or with absolute, constant geo-spatial coordinates in the environment (19), wherein the data structure (62) stores both the property data (27) observed in that volume element (28) and the confirmation value (61) for that volume element (28).

10. Method according to claim 9, wherein, by the processor circuit, the respective property data (27) of several spatial volume elements (28) (i.e. several “objects (29)”) are provided to a machine-learning-based object (29)-classifier process of the processor circuit, wherein the object (29)-classifier process performs an object (29) detection and / or object (29) classification for recognizing a predefined traffic infrastructure element and / or a dynamic traffic participant (23) in the environment (19) based on the property data (27) from the several spatial volume elements (28).

11. Method according to claim 10, wherein the method further comprises: signaling, by the processor circuit, the recognized traffic infrastructure element and / or traffic participant (23) together with its position in the environment (19) to a driver assistance functionality (13) that plans and / or executes a driving trajectory (16) that leads the vehicle (10) through the environment (19) and / or to a mapping functionality that generates and / or updates a dynamic map (18) of the environment (19).

12. Method according to any of the preceding claims, wherein the property data (27) comprises property values of different properties of the object (29), wherein the respective property value is calculated as a mean value of sensor (30) data obtained from successive measurements with the corresponding sensor (30) for that property and / or as a particle filtering result using the sensor (30) data as a filter input for a particle filter process of the processor circuit.

13. Method according to any of the preceding claims, wherein the tracking comprises odometric measurement of the movement of the vehicle (10) through the environment (19).

14. Processor circuit comprising instructions that when executed by the processor circuit cause the processor circuit to perform the following method steps confirming a spatial position (X,Y,Z) of an object (29) in an environment (19) of a vehicle (10):• generating, by the processor circuit, property data (27) describing at least one property of the object (29) using at least one predefined sensor (30) of the vehicle (10), whilethe object (29) is within a respective detection range (31) of the at least one predefined sensor (30), and• tracking, by the processor circuit, a relative motion (36) of the object (29) with regard to the vehicle (10) and associating with the property data (27) a spatial position (X,Y,Z) of the object (29) according to the tracked relative motion (36), and• obtaining, by the processor circuit, camera (45) images of the environment (19) outside the detection range (31) from at least one camera (45) of the vehicle (10) and• detecting, by the processor circuit, at least one laser light patch (49) in the camera (45) images, wherein each detected laser light patch (49) is associated with a respective spatial region (X',Y',Z') where the laser light patch is located in the environment (19) according to the camera (45) images, and is associated with the object passing through a laser beam, and• signaling, by the processor circuit, a confirmation value (61), if the respective spatial region (X',Y',Z') associated with the at least one laser light patch (49) matches the spatial position (X,Y,Z) associated with the property data (27), thus confirming that the property data (27) are still effective at the spatial position (X,Y,Z).

15. Motor vehicle (24) (10) comprising at least one sensor (30) with a respective detection range (31) directed into an environment (19) of the vehicle (10), at least one laser device (43), at least one camera (45) and a processor circuit, wherein the vehicle (10) is designed to perform a method according to any of the preceding method claims.

Images