Rain detection with an environmental sensor for point-by-point detection of a vehicle's surroundings, especially with a lidar-based environmental sensor

Description

[0001] The present invention relates to a method for detecting precipitation using an environmental sensor for point-by-point detection of the environment of a vehicle.

[0002] The present invention also relates to a driving assistance system for a vehicle comprising an environmental sensor for point-by-point detection of the vehicle's environment and a computing device, which are connected to each other via a data transmission link, wherein the driving assistance system is configured to perform the above method.

[0003] Precipitation encompasses all types of rain, snow, or hail that fall from the air to the ground in the form of drops, flakes, or grains of varying sizes. Precipitation can alter road surface properties or, more generally, the vehicle's handling characteristics. Furthermore, precipitation can impair a vehicle's perception of its surroundings, both for a human driver and for environmental sensors, such as those increasingly used in vehicle driver assistance systems to monitor the vehicle's environment. Therefore, it is crucial to reliably detect precipitation in order to, for example, adjust driving parameters based on the precipitation level.

[0004] In this context, exemplary methods and systems for detecting weather conditions, including wet surfaces, using vehicle-mounted sensors are known from US Patent 9,207,323 B2. One such method involves receiving laser data collected for a vehicle's environment. The method also includes determining laser data points associated with one or more objects in the environment and, based on laser data points not associated with the one or more objects in the environment, identifying an indication that a surface on which the vehicle is driving is wet.The method can further include receiving radar data acquired for the vehicle's surroundings, indicating the presence of one or more objects in the vehicle's environment, and identifying the indication that the surface on which the vehicle is driving is wet, based on laser data points not associated with the one or more objects in the environment indicated by the radar data. A similar lidar-based system is disclosed in WO2014168851.

[0005] The above-mentioned, state-of-the-art method is based on classifying points in a point cloud according to their object affiliation and drawing conclusions about the weather from this classification. This method is very complex, as it requires processing all points in the point cloud.

[0006] Based on the aforementioned prior art, the invention is therefore based on the objective of providing a method for detecting precipitation and a driving support system for carrying out the method, each of the type mentioned above, which enable efficient detection of precipitation with minimal effort.

[0007] The problem is solved according to the invention by the features of the independent claims. Advantageous embodiments of the invention are specified in the dependent claims.

[0008] According to the invention, a method for detecting precipitation according to claim 1 is thus provided.

[0009] According to the invention, a driving assistance system for a vehicle is also specified, comprising an environmental sensor for point-by-point detection of the vehicle's environment and a computing device, which are connected to each other via a data transmission link, wherein the driving assistance system is configured to perform the above method.

[0010] The basic idea of ​​the present invention is therefore to detect precipitation by monitoring a number of regions within the field of view of the environmental sensor. This has the advantage that only a portion of the environmental sensor's field of view needs to be considered, allowing the method to be carried out with minimal effort. Precipitation detection is based on the fact that, during precipitation, the environmental sensor detects a certain number of points in at least one region that cannot be assigned to any object. These points correspond to the precipitation. Accordingly, these points are neither assigned to an object nor detected as objects. By limiting the evaluation of the environmental points to at least one region, the processing can be carried out simply and efficiently.

[0011] The vehicle can be any type of vehicle driven by a human driver, or one capable of autonomous or semi-autonomous driving. The vehicle is preferably equipped with a driver assistance system that includes a corresponding environmental sensor and, based on this, creates, for example, an environmental map in which the points detected by the sensor are marked. Thus, the method according to the invention can be carried out particularly efficiently, since no separate detection of points in the vehicle's environment is required. According to the invention, only additional processing of already acquired data is then necessary.

[0012] The driving assistance system is either a standalone system designed to detect precipitation and display it to the driver, or to report the precipitation to another driving assistance system in the vehicle. Alternatively, the driving assistance system can be integrated with other driving assistance systems in the vehicle. The driving assistance system can have its own independent processing unit, or the processing unit can be a shared unit for performing various tasks for different driving assistance systems.

[0013] The computing unit can be an ECU (Electronic Control Unit), i.e., the vehicle's control unit. The computing unit comprises, for example, a processor and memory in which a program is stored or can be stored, which can then be executed on the computing unit's processor. At least partial implementation of the computing unit as an FPGA is possible.

[0014] The data transmission link can be implemented, for example, as a CAN bus commonly used in the automotive sector. In principle, however, other well-known bus systems can also be used. The processing unit can configure and / or control the environmental sensor via the data transmission link and also receive data points determined by the environmental sensor. Alternatively, the data points from the environmental sensor can be transmitted via the data transmission link to another processing unit, which, for example, performs object detection and / or generates an environmental map with the points detected by the environmental sensor. This data can then be transmitted from this other processing unit via the data transmission link to the processing unit of the driver assistance system.

[0015] The environmental sensor for point-by-point detection of the vehicle's surroundings can be designed in various ways. In an advantageous embodiment of the invention, the environmental sensor for point-by-point detection of the vehicle's surroundings is designed as a LiDAR-based environmental sensor. The vehicle's surroundings are detected using laser pulses that are reflected by objects in the vehicle's vicinity. Such reflection also occurs by precipitation, thereby generating corresponding environmental points from the LiDAR-based environmental sensor.

[0016] Alternatively, the environmental sensor for point-by-point detection of the vehicle's surroundings can be implemented as a radar-based sensor or as an optical camera. With a radar-based sensor, the detection process is similar to that of a LiDAR-based sensor, except that radar pulses are generated instead of laser pulses, and their reflections from objects in the vehicle's vicinity are detected. With an optical camera, the points are determined solely through image processing.

[0017] The detection of environmental points by the environmental sensor takes place within its field of view. Depending on the type of environmental sensor, the field of view can, for example, encompass an angle of 120° to 150° relative to a ground surface, which is a typical value for a LiDAR-based environmental sensor. Alternatively, the field of view can also encompass an angle of up to 180° or even more, as is common for optical cameras, especially wide-angle cameras. Radar-based environmental sensors can even have a field of view of up to 360°. In the vertical direction, the field of view of the environmental sensor is typically significantly smaller.

[0018] Selecting at least one region within the environmental sensor's field of view can, in principle, comprise one or more regions of arbitrary shape. A region in the direction of travel ahead of the vehicle is particularly relevant, as the vehicle will travel within this area and precipitation in this region can directly influence road surface properties or, more generally, the vehicle's handling characteristics on the road. The region can extend vertically across the entire field of view of the environmental sensor or only across a portion thereof. Each region is preferably designed such that its extent along the vertical direction remains constant. Selecting the at least one region within the environmental sensor's field of view only needs to be performed once, even if the process is repeated or carried out continuously.The step of selecting at least one region can therefore be carried out at any time in principle, i.e., also in a different sequence of individual procedural steps than the one mentioned here as an example.

[0019] Determining the point density of environmental points detected by the environmental sensor in at least one region can be done by determining a three-dimensional point density, in which case the volume of the region must be determined. Alternatively, a two-dimensional determination can be made by projecting the points in the region onto a base area of ​​the respective region and determining the point density in the base area.

[0020] Detecting precipitation by comparing the point density in at least one region with a predefined threshold is based, for example, on an empirically determined threshold. In principle, the threshold can be dynamically adjusted, for example, depending on different driving situations or environmental conditions.

[0021] In an embodiment of the invention, the method comprises checking the at least one region for the presence of an object, and the detection of precipitation by comparing the point density in the at least one region with a predetermined threshold value is performed in those regions where no object is located. If an object is present in a region, a large number of points inevitably lie within this region. However, these surrounding points cannot be attributed to precipitation and would therefore distort the detection of precipitation. To simplify the method, regions containing at least one object are therefore not used to determine precipitation.

[0022] In an advantageous embodiment of the invention, checking the at least one region for the presence of an object includes checking whether an object is present in all regions, and detecting precipitation when objects are present in all regions includes detecting precipitation based on a previous precipitation detection. Thus, if at least one object is present in all regions under consideration, no current precipitation detection can be performed based on comparing the point density in the at least one region with the predetermined threshold. Since the objects are often dynamic and moving while driving, the next precipitation detection is only performed when there is no object in at least one region. The previous precipitation state is maintained.Alternatively, a selection of at least one region can be repeated in order to obtain at least one region in which no object is present.

[0023] In an advantageous embodiment of the invention, selecting at least one region within the field of view of the environmental sensor comprises selecting a plurality of regions, and detecting precipitation by comparing the point density in the plurality of regions with a predetermined threshold value comprises detecting precipitation by determining a number of regions in which the comparison of the point density with the predetermined threshold value indicates precipitation. Thus, the number of regions with precipitation is compared with the number of regions without precipitation. When considering a plurality of regions for precipitation detection, it is first determined individually for each region whether precipitation is detected. Subsequently, for example, an average is calculated, or precipitation is detected by determining whether precipitation is detected in a specific, predetermined proportion of the regions.

[0024] In an advantageous embodiment of the invention, selecting at least one region within the field of view of the environmental sensor comprises selecting a plurality of regions, and detecting precipitation by comparing the point density in the plurality of regions with a predetermined threshold value comprises comparing the point density in the plurality of regions with the predetermined threshold value. Here, the point density is determined taking all regions into account; that is, the point density is determined jointly for all regions under consideration. Different configurations and sizes of the regions can be taken into account accordingly.

[0025] In an advantageous embodiment of the invention, selecting at least one region within the field of view of the environmental sensor comprises selecting the at least one region based on different sensitivities of the environmental sensor within that field of view. These different sensitivities can, for example, result from different detection accuracies. In principle, the different sensitivities can be properties of the corresponding environmental sensor. For example, a more precise angular resolution can be used in a central angular range in the horizontal plane than in peripheral regions of the field of view. Alternatively or additionally, the different sensitivities can result from a configuration or control of the corresponding environmental sensor.

[0026] InIn an advantageous embodiment of the invention, the method comprises an additional step for detecting a number of regions in which an object is located, and a step for performing a modified selection of at least one region within the field of view of the environmental sensor when the number of regions in which an object is located exceeds a predetermined threshold. In principle, any selection of regions can be made, provided that the selection is at least disjoint from the previous selection of regions. Thus, at least partial overlap of the regions is possible. In this way, the detection of precipitation can always be carried out reliably, regardless of the presence of objects.

[0027] In an advantageous embodiment of the invention, determining the point density of ambient points detected by the environmental sensor in at least one region comprises determining the point density by dividing the number of points in a respective region by a size of the respective region. This method step can be performed for both two-dimensional point density, where the points are projected onto a base area of ​​the region, and three-dimensional regions. The two-dimensional approach is particularly easy to perform. If the sizes of the individual regions do not change, these sizes can be determined once, and the method can then be carried out based on these values.

[0028] In an advantageous embodiment of the invention, determining a point density of environmental points detected by the environmental sensor in at least one region comprises determining a weighted point density, wherein the point density of each region is multiplied by a weighting factor and then the sum of the weighted point densities is divided by a number of regions. This weighting allows point densities in certain regions to be given greater or lesser weight in precipitation detection. For example, a region in the direction of travel can be given greater weight by the vehicle, since precipitation there can have a direct impact on the vehicle's driving characteristics.For example, areas where moisture is typically kicked up by vehicles ahead, so-called spray mist, can be given less consideration, as this is only an indirect indication of precipitation.

[0029] In an advantageous embodiment of the invention, the detection of precipitation by comparing the point density in at least one region with a predetermined threshold value comprises comparing the point density in at least one region with a plurality of predetermined threshold values, wherein the plurality of threshold values ​​indicate different precipitation intensities. Thus, not only can precipitation be detected as such, but a distinction can also be made, for example, between light rain and heavy rain, which may necessitate an individual modification or adjustment of the driving characteristics. This can further improve the vehicle's driving safety. For example, aquaplaning is more likely to occur in heavy rain. Visibility is also more severely impaired in heavy rain than in light rain, so a corresponding distinction can be helpful and improve driving safety.This applies to a human driver as well as various types of environmental sensors.

[0030] In an advantageous embodiment of the invention, comparing the point density in the at least one region with a plurality of predefined limit values ​​comprises comparing the point density in the at least one region with a plurality of predefined limit values ​​based on a look-up table. The look-up table can be determined, for example, depending on a vehicle type, an environmental sensor used, or other characteristics that may influence the detection of precipitation.

[0031] In an advantageous embodiment of the invention, the method comprises a step for determining the predetermined limit value based on sensor parameters of the environmental sensor. The sensor parameters can, for example, include an angular resolution of the environmental sensor. This resolution can vary for different regions or even within individual regions.

[0032] The invention is explained in more detail below with reference to the accompanying drawing and preferred embodiments. The features shown can represent an aspect of the invention, either individually or in combination. Features of different embodiments are transferable from one embodiment to another.

[0033] It shows Fig. 1 a schematic view of a vehicle according to a first, preferred embodiment with a driving assistance system comprising a LiDAR-based environmental sensor and a computing unit in a top view, Fig. 2 a flowchart of a method for detecting precipitation with an environmental sensor for point-by-point detection of a vehicle's environment in accordance with the first embodiment, Fig. 3 an exemplary representation of the environmental sensor made of Fig. 1 with its field of view, in which two regions are marked, in a top view and in a side view, Fig. 4 an exemplary representation of the environmental sensor made of Fig. 1 with its field of view, in which two regions are marked, and an object extending into one of the regions, in a top view and in a side view, and Fig. 5 an exemplary representation of the environmental sensor made of Fig. 1with its field of view, in which two regions are marked, and an object which extends into both regions, in a top view and in a side view.

[0034] The Figure 1 shows a vehicle 10 according to a first, preferred embodiment of the present invention.

[0035] Vehicle 10 is any vehicle 10 which is driven by a human driver, or which can drive autonomously or semi-autonomously.

[0036] The vehicle 10 is equipped with a driving assistance system 12, which includes an environmental sensor 14 for point-by-point detection of the vehicle 10's surroundings and a computing unit 16. The environmental sensor 14 and the computing unit 16 are connected to each other via a data transmission link 18.

[0037] The driving assistance system 12 is implemented here as an independent driving assistance system 12 to detect precipitation and display it to the driver or to report the precipitation to another driving assistance system 12 of the vehicle 10.

[0038] The environmental sensor 14 is implemented here as a LiDAR-based environmental sensor 14. The environment of the vehicle 10 is detected using laser pulses that are reflected by objects 26 in the vicinity of the vehicle 10. Such reflection also occurs by precipitation, thereby generating corresponding environmental points from the LiDAR-based environmental sensor 14.

[0039] In this embodiment, the computing unit 16 is an independent computing unit 16 of the vehicle assistance system 12. The computing unit 16 is designed as an ECU (Electronic Control Unit), i.e., a control unit of the vehicle 10. The computing unit 16 comprises a processor and a memory in which a program is stored that can be executed on the processor of the computing unit 16.

[0040] In this embodiment, the data transmission link 18 is implemented as a CAN bus, which is common in the automotive sector. The computing unit 16 can configure and / or control the environmental sensor 14 via the data transmission link 18 and also receive data points determined by the environmental sensor 14.

[0041] Below is a method for detecting precipitation using an environmental sensor 14 for point-by-point detection of the environment of a vehicle 10 in accordance with the first embodiment. The method is presented as a flowchart in Figure 2 The method is described and is carried out using the driving assistance system 12 of the vehicle 10 of the first embodiment. The method is further described with reference to the Figures 3 to 5 described.

[0042] The procedure begins with step S100, which involves capturing environmental points with the environmental sensor 14 within its field of view 20. The field of view 20 of the single LiDAR-based environmental sensor 14 typically covers an angular range of 120° to 150° relative to a ground surface 28. In the horizontal direction, the field of view 20 has a smaller angular range, for example, 30° to 40°.

[0043] In step S110, a plurality of regions 22, 24 are selected within the field of view 20 of the environmental sensor 14. In this embodiment, a first region 22 and a second region 24 are selected, both located in front of the vehicle 10 in the direction of travel 30. The two regions 22, 24 have different shapes and are adjacent to each other. The first region 22, which is closer to the vehicle 10, has a triangular shape horizontally. The second region 24 adjoins it and forms an extension of the triangular shape of the first region 22. In the vertical direction, the two regions 22, 24 extend over the entire field of view 20 of the environmental sensor 14, with the dimensions of the two regions 22, 24 remaining constant along the vertical direction.

[0044] The first and second regions 22, 24 are selected such that they are located in the same angular segment starting from the environmental sensor 14, so that the sensitivity of the environmental sensor 14 is identical in the first and second regions 22, 24.

[0045] In step S120, the two regions 22 and 24 are checked for the presence of an object 26. It is checked in which of the two regions 22 and 24 an object 26 is present. Figures 3 to 5 This is illustrated by example, showing that an object 26 is not present in either region 22, 24, as in Figure 3 as depicted, is only present in the second region 24, as in Figure 4 is depicted, or is present in both regions 22, 24, as in Figure 5 is shown.

[0046] If an object 26 is present in both regions 22 and 24, the procedure ends at this point and returns to step S100. Precipitation detection is aborted because neither region 22 or 24 can be evaluated with respect to environmental points detected by the environmental sensor 14. The result for precipitation detection is a previous precipitation detection.

[0047] In step S130, which is optional in the described procedure, a number of regions 22, 24 containing an object 26 are identified. If the number of regions 22, 24 containing an object 26 exceeds a predefined limit, a new selection of regions 22, 24 is made. The procedure then returns to step S110.

[0048] In step S140, the point density of environmental points detected by the environmental sensor 14 in regions 22 and 24, where no objects 26 are located, is determined. A three-dimensional point density of the respective regions 22 and 24 is calculated. For this purpose, the number of detected environmental points in each of the regions 22 and 24 is determined and divided by the volume of the respective region 22 and 24. The volume of regions 22 and 24 is determined only during the first iteration of the procedure and is then stored for subsequent iterations.

[0049] In an alternative embodiment, determining a point density of environmental points detected by the environmental sensor 14 in regions 22, 24 comprises determining a weighted point density, wherein a point density of each region 22, 24 is multiplied by a weight factor and then the sum of the weighted point densities is divided by a number of regions 22, 24.

[0050] Alternatively, a two-dimensional determination of the point density can be carried out by projecting the recorded environmental points in regions 22, 24 onto the ground surface 28 of the respective region 22, 24, and determining the point density in the ground surface 28.

[0051] Step S150 involves detecting precipitation by comparing the point density in regions 22 and 24, where no object 26 is located, with a plurality of predefined threshold values, where the plurality of threshold values ​​indicate different precipitation intensities. The plurality of predefined threshold values ​​are based on a lookup table. The lookup table is determined here, by way of example, depending on a vehicle type, an environmental sensor 14 used, sensor parameters of the environmental sensor 14, or other characteristics. In this embodiment, the sensor parameters include an angular resolution of the environmental sensor 14.

[0052] Specifically, the point density in regions 22 and 24 is averaged. This averaged point density is then compared to the limit values. Thus, the point density is determined taking all regions 22 and 24 into account; that is, the point density is determined jointly for all regions 22 and 24 in which no object 26 is located. Different configurations and sizes of regions 22 and 24 can be considered in this process.

[0053] The detection of precipitation by comparing the point density in regions 22 and 24, where no object 26 is located, with the predefined threshold values ​​is based on empirically determined limits. In an alternative configuration, the threshold values ​​are dynamically adjusted, for example, depending on different driving situations or environmental conditions. Reference symbol list

[0054] 10 Vehicle 12 Driver assistance system 14 Environmental sensor, LiDAR-based environmental sensor 16 Computing unit 18 Data transmission link 20 Field of view 22 First region 24 Second region 26 Object 28 Ground surface 30 Direction of travel

Claims

1. Method for detecting precipitation by means of an environment sensor (14) for capturing an environment of a vehicle (10) point by point, wherein the environment sensor (14) for capturing an environment of the vehicle (10) point by point is embodied as a LiDAR-based environment sensor (14), comprising the steps of capturing environment points by means of the environment sensor (14) within the latter's field of view (20), selecting at least one region (22, 24) within the field of view (20) of the environment sensor (14), determining a point density of environment points captured by means of the environment sensor (14) in the at least one region (22, 24), detecting precipitation by comparing the point density in the at least one region (22, 24) with a predefined limit value, checking the at least one region (22, 24) for the presence of an object (26), and detecting precipitation by comparing the point density in the at least one region (22, 24) with a predefined limit value in such regions (22, 24) in which no object (26) is located, wherein regions (22, 24) in which at least one object (26) is located are not used for ascertaining precipitation.

2. Method according to the preceding Claim 1, characterized in that checking the at least one region (22, 24) for the presence of an object (26) comprises checking whether an object (26) is present in all regions (22, 24), and detecting precipitation when objects (26) are present in all regions (22, 24) comprises detecting precipitation on the basis of a previous detection of precipitation.

3. Method according to either of the preceding claims, characterized in that selecting at least one region (22, 24) within the field of view (20) of the environment sensor (14) comprises selecting a plurality of regions (22, 24), and detecting precipitation by comparing the point density in the plurality of regions (22, 24) with a predefined limit value comprises detecting precipitation by determining a number of regions (22, 24) in which the comparison of the point density with the predefined limit value indicates precipitation.

4. Method according to any of the preceding claims, characterized in that selecting at least one region (22, 24) within the field of view (20) of the environment sensor (14) comprises selecting a plurality of regions (22, 24), and detecting precipitation by comparing the point density in the plurality of regions (22, 24) with a predefined limit value comprises comparing the point density in the plurality of regions (22, 24) with the predefined limit value.

5. Method according to any of the preceding claims, characterized in that selecting at least one region (22, 24) within the field of view (20) of the environment sensor (14) comprises selecting the at least one region (22, 24) depending on different sensitivities of the environment sensor (14) in the field of view (20).

6. Method according to any of the preceding claims, characterized in that the method comprises an additional step for capturing a number of regions (22, 24) in which an object (26) is located, and the method comprises a step for carrying out a modified selection of at least one region (22, 24) within the field of view (20) of the environment sensor (14) if the number of regions (22, 24) in which an object (26) is located is greater than a predefined limit value.

7. Method according to any of the preceding claims, characterized in that determining a point density of environment points captured by means of the environment sensor (14) in the at least one region (22, 24) comprises determining the point density by dividing a number of points of a respective region (22, 24) by a size of the respective regions (22, 24).

8. Method according to any of the preceding claims, characterized in that determining a point density of environment points captured by means of the environment sensor (14) in the at least one region (22, 24) comprises determining a weighted point density, wherein a point density of each region (22, 24) is multiplied by a weight factor and then the sum of the weighted point densities is divided by a number of regions (22, 24).

9. Method according to any of the preceding claims, characterized in that detecting precipitation by comparing the point density in the at least one region (22, 24) with a predefined limit value comprises comparing the point density in the at least one region (22, 24) with a plurality of predefined limit values, wherein the plurality of limit values specify different intensities of precipitation.

10. Method according to the preceding Claim 9, characterized in that comparing the point density in the at least one region (22, 24) with a plurality of predefined limit values comprises comparing the point density in the at least one region (22, 24) with a plurality of predefined limit values on the basis of a look-up table.

11. Method according to any of the preceding claims, characterized in that the method comprises a step for ascertaining the predefined limit value on the basis of sensor parameters of the environment sensor (14).

12. Driving assistance system (12) for a vehicle (10) comprising an environment sensor (14) for capturing an environment of the vehicle (10) point by point and a computing device (16), which are connected to one another via a data transmission connection (18), wherein the environment sensor (14) for capturing an environment of the vehicle (10) point by point is embodied as a LiDAR-based environment sensor (14) and wherein the driving assistance system (12) is designed to carry out the method according to any of the preceding claims.