Environmental sensing system for a motor vehicle

Abstract

The invention relates to an environmental sensing system (10) for a motor vehicle, the environmental sensing system having a plurality of sensors (12-20) sensitive to electromagnetic radiation in different regions of the electromagnetic spectrum, the environmental sensing system further having a corresponding analysis and processing module (22) for localizing and / or classifying objects present in the environment based on data provided by the sensors, characterized in that the environmental sensing system has a model module (26, 28) in which an environmental model is stored, the environmental model including, in addition to 3D position data (34) of the objects, data (36) on the spectral reflection characteristics of the object surfaces, wherein the model module is capable of providing this data to the analysis and processing module (22).

Description

Technical Field

[0001] The present invention relates to an environmental sensing system for a motor vehicle, which environmental sensing system has a plurality of sensors and corresponding analysis and processing modules, the plurality of sensors being sensitive to electromagnetic radiation in different regions of the electromagnetic spectrum, and the analysis and processing modules being configured to localize and / or classify objects present in the environment based on data provided by the sensors. Background Art

[0002] In driver assistance systems for motor vehicles, it is important to sense the environment of the vehicle as precisely as possible with suitable sensing means. In the process of increasingly automated vehicle guidance, ever more stringent requirements are placed on the accuracy and reliability of environmental sensing systems.

[0003] It is known to use different types of sensors operating in different regions of the electromagnetic spectrum, such as long-range radar sensors and short-range radar sensors with different radar frequencies, lidar sensors, and optical cameras. If the data obtained from the various sensors are fused with one another, then an image of the sensed environment can be made complete.

[0004] However, the problem is that the radiation emitted or reflected by an object may be reflected at the surface of other objects and then reach the sensor in various ways, whereby pseudo-objects (Scheinobjekt) that do not actually exist are fabricated in the sensor. Examples in this regard are, for example, the reflection of radar radiation on the road surface or guardrails, the reflection of visible light on shop windows, etc. The pseudo-objects fabricated in this way are usually difficult to distinguish from real objects, such that an unrealistic image of the environment is obtained. Summary of the Invention

[0005] The object of the present invention is to provide an environmental sensing system that can better distinguish real objects from artifacts determined by reflections.

[0006] According to the present invention, this object is solved by a model module storing an environmental model, the environmental model including, in addition to 3D position data of the objects, data on the spectral reflection characteristics of the object surfaces, wherein the model module can provide this data to the analysis and processing modules.

[0007] Then, based on the model stored in the model module, the beam propagation in the wavelength range (to which the relevant sensors are sensitive) can be simulated and thus the occurrence of pseudo-objects generated by multiple reflections can be predicted. Based on this information, it is easier to filter out pseudo-objects when analyzing and processing sensor data, thereby significantly reducing the likelihood of misinterpretation.

[0008] Advantageous configurations and extensions of the present invention are described in preferred embodiments.

[0009] Data can be stored in the environmental model, where the data represents the positions of objects existing in the environment and represents the geometry with a more or less high spatial resolution. Then, based on this data, the positions, geometries, and orientations of the object surfaces where potential reflections may occur can also be determined. Additionally, for each of these surfaces, at least one parameter is stored, which characterizes the reflection characteristics of the surface in one or more wavelengths used in the sensor. For example, a reflection coefficient and / or a glossiness parameter can be stored for each surface and for each frequency of interest, where the glossiness parameter describes the proportional relationship between the directed reflection and the diffuse reflection.

[0010] In one embodiment, the model module can be implemented in a vehicle. Then, the data for modeling is provided by the sensors for sensing the environment, and the data is supplemented, if necessary, by the "prior knowledge" about the typical characteristics of the objects. For example, if a noise protection wall, a guardrail, or the front of a building is recognized by means of a camera system, the positions and orientations of the reflective surfaces can be identified based on the camera data and input into the model. At the same time, the reflection characteristics of visible light and the reflection characteristics of radar radiation in the case of the frequencies of interest can be estimated, and the reflection characteristics can also be input into the model, so that the artifacts determined by the reflection can be predicted.

[0011] Conversely, the data provided by a radar sensor or a lidar sensor can be used to accurately determine the distances of the objects, so as to obtain the distance information that is sometimes difficult to obtain with the required accuracy by means of a camera system or a stereo camera. By accessing the environmental model, the distance data can be provided to the analysis and processing module of the camera system, and the distance data facilitates the interpretation of the camera image.

[0012] The environmental model can be dynamically matched to the corresponding position of the vehicle equipped with the environmental sensing system and the positions of moving objects (such as other vehicles).

[0013] However, in another embodiment, the model module can also be implemented in a server fixed in position outside the vehicle, and the server communicates wirelessly with the environmental sensing system in the vehicle. Such a position-fixed model module is particularly suitable for storing the environmental model for a given section in a traffic network and can provide its data to the environmental sensing systems of multiple vehicles. The advantage is that in this case, the surface reflection characteristics, positions, and geometries of the objects can be measured or obtained from other sources and manually input into the model, thereby achieving a higher accuracy of the model. However, since these data can only be updated from time to time at most, in this case, the model is a relatively static model.

[0014] Particularly advantageously, the two embodiments are combined with each other such that the analysis processing module can access the dynamic model in the local model module on the one hand and the static but more accurate model in the fixedly located module on the other hand. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the following, the embodiments are explained in more detail on the basis of the drawings. The drawings show:

[0016] Figure 1 A block diagram of the environmental sensing system of the present invention;

[0017] Figure 2 An example of the data structure of the environmental model;

[0018] Figure 3 For running in an environmental sensing system according to Figure 1 A flowchart of a routine for predicting reflection artifacts in an environmental sensing system, the routine being used for predicting reflection artifacts;

[0019] Figure 4 and Figure 5 An illustration for explaining the traffic situation to which the present invention is applied. DETAILED DESCRIPTION OF THE INVENTION

[0020] In Figure 1 the outline of a motor vehicle 10 is schematically shown, in which various different sensors for sensing the environment are installed. In the example shown, the sensors are remote radar sensors 12, 14, short-range radar sensors 16, a stereo camera system, and a lidar sensor 20. The remote radar sensors operate, for example, at a frequency of 76 GHz and are particularly used for sensing the traffic in front and behind. The short-range radar sensors operate, for example, at a frequency of 24 GHz and are installed at the four corners of the vehicle in order to monitor the closer vehicle environment. The stereo camera system has two optical cameras 18 mounted on the front side of the vehicle. The lidar sensor is used for performing a panoramic surveillance of the environment by means of lidar beams having a certain frequency in the visible light region or the infrared region of the spectrum. Each of these sensors is assigned an analysis processing module 22 for pre-analyzing the data provided by the sensor. Then, the results of this pre-analysis are transmitted to a central analysis processing unit 24, where the results are further processed and fused with each other in order to obtain as complete an image of the vehicle environment as possible, which then forms the basis for various different driver assistance functions or autonomous vehicle control functions.

[0021] The local model module 26 is implemented in the central analysis and processing unit 24, and an environmental model is stored in the local model module, which describes the positions and geometries of the objects located by the vehicle's own sensors in the current environment of the vehicle 10. The model is continuously updated according to the data provided by the analysis and processing module 22.

[0022] In the illustrated example, the central analysis and processing unit 24 communicates wirelessly with a position-fixed model module 28, which is implemented, for example, in a server outside the vehicle, and a static model of traffic infrastructure and static objects in the section of the road traversed by the vehicle 10 is stored in the position-fixed model module.

[0023] Figure 2 An example of the data structure of the environmental model 30 stored in the model module 26 or 28 is shown. The model includes a list of objects 32 present in the environment.

[0024] In the case of the local model module 26, examples of such objects are, for example: "the vehicle driving directly in front", "the vehicle in the adjacent lane", "the pedestrian at the edge of the carriageway", "the guardrail", "the carriageway surface", etc. Each of these objects is located by at least one of the sensors 12 to 20 and classified, if necessary, by comparing the data of multiple sensors.

[0025] In the case of the position-fixed model module 28, the object 32 can be, for example, the buildings on the right and left sides of the carriageway, or can also be the guardrail, the carriageway surface, etc.

[0026] For each of the objects 32, 3D position data 34 is stored in the environmental model, which identifies the position of the relevant object and, if necessary, its geometry. In the local model module 26, in the case of the vehicle driving in front, the position data can be, for example, the distance and direction angle (azimuth angle and possibly elevation angle) measured by the radar sensor. In the case of the guardrail, the position data 24 can be composed, for example, of the position and orientation of the main surface of the guardrail, while in the case of the carriageway surface, the position data can include the carriageway slope as a function of the distance. In the local model module 26, the position data can be calculated, for example, based on camera data, while in the position-fixed model module 28, these data can be directly input. In the case of a building having an approximately cubic shape stored in the position-fixed model module 28, the position data 34 can be the coordinates of the corners of the building in the position-fixed global coordinate system.

[0027] With the aid of the 3D position data 34, it is possible for each object 32 to identify surfaces that, depending on their apparent wavelength, can reflect electromagnetic radiation more or less well. Of particular interest here are those surfaces that can reflect the radiation into one of the sensors of the vehicle 10. For each of these surfaces, a set of data 36 is stored in the environment model 30, which (for example, based on known material properties or, if necessary, based on previous measurements or object classifications) describes the spectral reflection characteristics of the relevant surface. In the example shown, reflection coefficients ρ and glossiness parameters γ are stored for three different frequencies f1 (76 GHz), f2 (24 GHz), and f3 (the frequency of visible light used by the lidar sensor 20). Then, based on this data, it is possible to calculate for each surface under consideration how electromagnetic radiation of the relevant wavelength is reflected and / or scattered on the surface. Then, based on the calculation results, it can be determined whether the reflected or scattered radiation reaches one of the vehicle's own sensors and creates a pseudo-object there fictitiously. It is also possible to predict the intensity of the signal that describes the pseudo-object within certain boundaries. This makes it easier in the analysis processing module 22 or the central analysis processing unit 24 to distinguish real objects from reflection-based pseudo-objects.

[0028] Figure 3 Shows the basic steps of a method by which, for example, the occurrence of pseudo-objects or ghosts (Geisterbild) is predicted in the central analysis processing unit 24. In step S1, individual objects are identified based on the signals provided by one or more of the analysis processing modules. Then, in this step, the object is searched for in the environment model 30, and in step S2, the 3D position data 34 and the data 36 on the reflection characteristics of the object are read, where, if necessary, the position data is converted into a vehicle-fixed coordinate system. Then, in step S3, the reflection path of the beam emitted from the located object and reflected on the following surface is calculated: the characteristics of which have been read in step S2. Then, in step S4, those reflection paths that lead to one of the vehicle's own sensors 12 to 20 are selected, and based on the reflection characteristics applicable to the corresponding frequency, the signal that creates a ghost in the relevant sensor is predicted.

[0029] In Figure 4In this case, the following situation is shown as an example: In this situation, the vehicle 10 is traveling on the lane 38, on which a noise protection wall 40 is installed on one side. The radar sensor 12 of the vehicle 10 emits a radar beam 42, which is reflected at the rear of the vehicle 44 (object) traveling in front and returns directly to the radar sensor 12 as a reflected beam 46. However, since the rear of the vehicle 44 is not completely flat, a part of the radar radiation is also reflected in the direction of the noise protection wall 40 and then reflected again at the noise protection wall, so that the reflected beam 48 reaches the radar sensor 12 in a detour and a false object 50 is fictitiously created there, which has the same relative speed as the vehicle 44 but has a slightly greater distance and is seen at a different direction angle.

[0030] According to the process shown in Figure 3 the process can be simulated so that the false object 50 can be predicted and correctly interpreted as a non-real object.

[0031] In the example shown, the noise protection wall 40 has a rough sound-absorbing structure 52 in a section (on the left side in Figure 4 ), while the subsequent section of the noise protection wall has a smooth surface 54. On the rough structure 52, the radar radiation is mostly diffusely scattered, so that mostly weak reflection signals are generated, which hardly stand out from the noise background. In contrast, if the vehicle reaches the section of the noise protection wall 40 with the smooth surface 54, then a distinct reflection signal suddenly appears, which then has to be quickly and correctly interpreted by the environment sensing system of the vehicle 10. The change in the structure on the noise protection wall 40 can be recognized by means of the camera 18 on the vehicle 10, so that the sudden appearance of the reflection signal can be predicted. Similarly, the prediction can also be made based on the model stored in the position-fixed model module 28.

[0032] In a similar way, the reflection of visible light can also be predicted, for example, the reflection on the front of a shop window.

[0033] Another example is the prediction of the reflection of radar beams on the lane surface. These reflections can be calculated based on the stored lane gradient and / or based on the lane gradient estimated by means of the camera 18. In addition, for example, the wet lane surface and the dry lane surface can be distinguished based on the camera image, and then the reflection characteristics of the radar radiation can be correspondingly matched in the environment model 30.

[0034] Figure 5An example is shown in which the rear part 56 of a truck 58 traveling ahead is located by means of the camera 18 and the lidar sensor 20 of the vehicle 10. The rear part 56 consists, for example, of a tilted loading platform with a flat, almost structureless surface. This makes it difficult to measure distances by means of a stereo camera system, since for distance determination the parallax displacement of the structures detected in the field of view of the camera must be determined.

[0035] Since the vehicle is currently turning, the rear part 56 is not at right angles to the direction of travel of the vehicle 10. Since the rear part 56 has a high gloss value in visible light, the lidar sensor 20 only provides a signal when its beam reaches the rear part 56 at a right angle. Therefore, it is not possible to recognize with the help of the lidar sensor 20 that the rear wall is actually an object that extends significantly further. Based on the data of the camera 18, the following information can now be added to the environment model 30: the rear wall 26 is largely flat and structureless and has a high gloss value. On the contrary, the exact value of the distance between the rear part 56 and the vehicle 10 can be explained based on the data of the lidar sensor. By now integrating this information, it can be clearly determined that the object seen by the camera 18 is the same rear part 56 that is also located by the lidar sensor 20. On the one hand, due to the high gloss value of the rear part, it can be expected that the lidar sensor only locates this rear part as a point-shaped object, although this rear part actually extends further. In contrast, the distance information provided by the lidar sensor allows targeted searching in the camera image for difficult-to-recognize structures whose parallax has a value that matches the measured distance, so that this distance measurement can also be verified with the aid of the camera system.

[0036] In a situation where the truck 58 is stationary and the loading platform is being tilted down, the increasing inclination of the rear part 56 leads to a sudden disappearance of the signal of the lidar sensor 20. This causes a search for a change in the contour shape of the loading platform in the camera image and thus determines that the loading platform is being tilted down and that the risk of collision may therefore be increasing.

Claims

1. An environment sensing system for a motor vehicle (10), having a plurality of sensors (12 to 20) and corresponding analysis and processing modules (22), the sensors being sensitive to electromagnetic radiation in different regions of the electromagnetic spectrum, the analysis and processing modules being adapted to localize and / or classify objects (32, 44, 58) present in the environment on the basis of data provided by the sensors, characterized in that The environmental sensing system has model modules (26, 28), in which an environmental model (30) is stored. The environmental model contains, in addition to 3D position data (34) of the objects (32, 44, 58), data (36) on the spectral reflection characteristics of the surfaces of the objects. The model modules are capable of providing this data to the analysis and processing module (22), and the beam propagation in such a wavelength range is simulated according to the model stored in the model modules: the sensor is sensitive in the wavelength range in order to predict the occurrence of pseudo-objects caused by multiple reflections.

2. The environmental sensing system according to claim 1, in which the model module (26) is implemented locally in the motor vehicle (10).

3. The environmental sensing system according to claim 1, in which the model module (28) is implemented in a position-fixed server, which communicates wirelessly with the analysis and processing device (24) in the motor vehicle (10).

4. The environmental sensing system according to claim 1, having a local model module (26) implemented in the motor vehicle and a position-fixed model module (28) implemented outside the motor vehicle.

5. The environmental sensing system according to any one of claims 1 to 4, having an analysis and processing device (24), which is configured to construct the environmental model (30) according to the data generated by the analysis and processing module (22).

6. The environmental sensing system according to any one of claims 1 to 4, having an analysis and processing device (24), the analysis and processing device being configured to calculate a reflection path of electromagnetic radiation based on data stored in the environmental model (30), the electromagnetic radiation starting from a located object and reaching one of the sensors (12 to 20) of the environmental sensing system due to reflection on the surface of another object, wherein, The analysis and processing device (24) is also configured to predict, according to the calculated reflection paths, signals that can be interpreted as object signals in the analysis and processing module (22) and to identify the objects corresponding to the signals as pseudo-objects.

Images