METHOD AND DEVICE FOR DETERMINING OCCURRENT AREAS IN THE VEHICLE SURROUNDINGS OF A VEHICLE

Description

[0001] The invention relates to a method and a device for determining masking areas in the vehicle environment of a vehicle, and in particular to a driver assistance system in which masking areas which limit the field of view of optical environmental sensors of the driver assistance system are determined.

[0002] Vehicles are increasingly equipped with driver assistance systems that support the driver in performing driving maneuvers. Such driver assistance systems feature display units that visually show the vehicle's surroundings to the driver. The vehicle cameras, which generate camera images of the vehicle's surroundings, transmit these images or image data to a data processing unit, which projects the image data onto a predefined projection surface to display it to the driver on a display. In addition to vehicle cameras and optical sensor units, driver assistance systems also feature other environmental sensors, such as ultrasonic sensors.

[0003] In conventional driver assistance systems, the presence of obstacles in the vehicle's surroundings, such as other vehicles parked nearby, can lead to disruptive image distortions in the displayed image of the vehicle's surroundings. These image distortions are caused by obscuration areas that limit the field of view of the driver assistance system's optical environmental sensors.

[0004] DE 10 2014 107 156 A1 discloses a system for providing an improved perspective view of an area in the front of a vehicle. The system comprises a first camera on the left front side of the vehicle and a second camera on the right front side of the vehicle as environmental sensors. Furthermore, the received images are evaluated to detect obstacles in the vehicle's surroundings. Furthermore, areas hidden behind projections of the vehicle and extending in front of the camera lenses are calculated and used to create the image.

[0005] DE 10 2013 019 145 A1 discloses a driver assistance system for a motor vehicle that detects the surroundings of the motor vehicle using optical environment sensors. Furthermore, the surroundings are divided into sub-areas, with relevant sub-areas that may be affected by shadowing. Measures are taken, such as driver interventions or chassis adjustments, to detect non-detected sub-areas.

[0006] US 2013 / 245877 A1 describes a driver assistance system with environmental sensors that are evaluated to detect obstacles in the vehicle's surroundings. Furthermore, depending on the detected obstacles, obscured areas in the vehicle's surroundings are determined that are obscured by obstacles and limit the field of vision. Various measures are also taken, such as suitable driving maneuvers, sensor fusion, or the provision of movable mounts for the sensors to make such obscured areas visible.

[0007] It is therefore an object of the present invention to provide a method and a device for determining such masking areas in the vehicle environment of a vehicle.

[0008] This object is achieved according to the invention by a device having the features specified in claim 1.

[0009] The invention therefore creates a driver assistance system for a vehicle with Environmental sensors that sense the vehicle's surroundings and with a data processing unit that evaluates sensor data from the environmental sensors to detect obstacles in the vehicle's surroundings, wherein, depending on detected obstacles, masking areas in the vehicle's surroundings are determined that are masked by the obstacles and limit a field of view of optical environmental sensors of the driver assistance system, and wherein the data processing unit calculates the masking areas caused by the detected obstacles depending on a relative speed between the detected obstacles and the vehicle.

[0010] The masking areas caused by the detected obstacles are preferably determined as a function of the relative position of the obstacle in question to an optical environmental sensor of the driver assistance system and / or as a function of the extent or size of the obstacle in question.

[0011] In one possible embodiment of the driver assistance system according to the invention, the data processing unit processes image sensor data originating from optical environmental sensors, in particular vehicle cameras, in the determined masking areas.

[0012] In one possible embodiment of the driver assistance system according to the invention, the determined masking areas are processed by the data processing unit by filtering the image sensor data through a filter.

[0013] In a further possible embodiment of the driver assistance system according to the invention, the data processing unit covers the masking areas caused by the detected obstacles with texture surfaces.

[0014] In a further possible embodiment of the driver assistance system according to the invention, the data processing unit additionally calculates the masking areas caused by the detected obstacles as a function of a projection surface used for image display, in particular as a function of a static two-dimensional floor surface or as a function of a three-dimensional bowl-shaped projection surface.

[0015] Furthermore, a control circuit is provided which controls optical environmental sensors, in particular vehicle cameras of the driver assistance system, depending on the masking areas caused by the detected obstacles.

[0016] The control circuit switches between different optical environmental sensors of the driver assistance system depending on the detected masking areas.

[0017] In a further possible embodiment of the driver assistance system according to the invention, the optical environmental sensors have vehicle cameras, in particular fisheye cameras, each of which has a predetermined field of view.

[0018] In a further possible embodiment of the driver assistance system according to the invention, the data processing unit calculates the masking areas caused by the detected obstacles as a function of the fields of view of the affected optical environmental sensors.

[0019] The invention further provides a method for determining masking areas with the features specified in claim 7.

[0020] The invention therefore provides a method for determining masking areas in the vehicle environment of a vehicle, comprising the steps: Evaluating sensor data generated by environmental sensors of the vehicle to detect obstacles in the vehicle environment of the vehicle and calculating, depending on the detected obstacles, masking areas that are masked by the obstacles and limit a field of view of the optical environmental sensors of the vehicle, wherein the masking areas caused by the detected obstacles are calculated depending on a relative speed between the detected obstacles and the vehicle.

[0021] In one possible embodiment of the method according to the invention, the image sensor data originating from optical environmental sensors are processed, in particular filtered.

[0022] In a further possible embodiment of the method according to the invention, the masking areas caused by the detected obstacles are covered with texture surfaces.

[0023] In a further possible embodiment of the method according to the invention, the masking areas caused by the detected obstacles are calculated as a function of a two- or three-dimensional projection surface used for image display.

[0024] In a further possible embodiment of the method according to the invention, optical environmental sensors, in particular vehicle cameras, are controlled depending on the determined masking areas.

[0025] Furthermore, in the method according to the invention, switching takes place between different optical environmental sensors, in particular vehicle cameras, depending on the determined masking areas.

[0026] In a further possible embodiment of the method according to the invention, the masking areas caused by the detected obstacles are calculated as a function of the predetermined fields of view of the relevant optical environmental sensors.

[0027] Possible embodiments of the driver assistance system according to the invention and of the method according to the invention for determining masking areas are explained in more detail below with reference to the attached figures.

[0028] They show: Figure 1 shows a schematic representation of an embodiment of the driver assistance system according to the invention; Figure 2 shows a block diagram illustrating an embodiment of the driver assistance system according to the invention; Figure 3 shows a schematic representation explaining the functioning of the driver assistance system according to the invention; Figure 4 shows a flowchart illustrating an embodiment of a method according to the invention for determining masking areas in the vehicle environment of a vehicle; Figure 5 shows a further flowchart illustrating a further embodiment of the method according to the invention.

[0029] Figure 1shows a schematic representation of a vehicle F having a driver assistance system according to the invention. In the illustrated embodiment, optical environmental sensors are mounted on various sides of the body of the vehicle F, which sensors detect the vehicle environment FU of the vehicle F. The optical environmental sensors can, for example, be vehicle cameras, which provide vehicle images of the vehicle environment. Figure 1 In the exemplary embodiment shown, the driver assistance system 1 of the vehicle F has four vehicle cameras 2-1, 2-2, 2-3, 2-4. The first vehicle camera 2-1 is mounted on the front of the body of the vehicle F and has a field of view or a field of view FOV1, as shown in Figure 1shown. Furthermore, there is a vehicle camera on the left and right side of the vehicle body of the vehicle F, which optically captures the lateral vehicle surroundings of the vehicle F. The vehicle camera 2-2 mounted on the left side of the vehicle F has a field of view or a field of view FOV2. The right vehicle camera 2-3 captures the part of the vehicle surroundings FU located to the right of the vehicle F and has a field of view FOV3, as shown in Figure 1 Furthermore, a vehicle camera 2-4 with a field of view FOV4 is provided on the rear of the vehicle F. The four optical environmental sensors 2-1 to 2-4 can, in one possible embodiment, be fisheye cameras with a relatively wide field of view FOV of more than 170°. As can be seen in Figure 1can detect, the fields of view (FOV) of the various vehicle cameras 2-i of the driver assistance system 1 may overlap. The various vehicle cameras 2-i are connected to a data processing unit 4 of the driver assistance system 1 via signal lines 3-1, 3-2, 3-3, 3-4, for example via a signal line bus or vehicle bus. Sensor data, in particular camera images, are transmitted to the data processing unit 4 via the signal lines 3-i. The data processing unit 4 evaluates the sensor data of the environmental sensors, in particular the Figure 1illustrated optical environmental sensors or vehicle cameras 2-i, for detecting obstacles H in the vehicle environment of the vehicle F. For this purpose, the data processing unit 4 has a processor which evaluates the sensor data. The sensor data is preferably processed in real time. In addition to the optical environmental sensors 2-i, the driver assistance system 1 can also have further environmental sensors, for example ultrasonic sensors. These further environmental sensors also provide sensor data which can be evaluated by the data processing unit 4 for detecting obstacles H in the vehicle environment of the vehicle F. In the Figure 1 In the example shown, there are two obstacles H1, H2 in the surroundings of vehicle F, for example a wall or building. Figure 1In the example shown, the obstacle H1 is located in the field of view FOV1 of the front vehicle camera 2-1. The second obstacle H2 is located partly in the field of view of the front vehicle camera 2-1 and partly in the field of view of the left vehicle camera 2-2. Based on the received sensor data, the obstacles H1, H2 are detected in the vehicle environment. This sensor data can be used by the Figure 1 illustrated vehicle cameras 2-i and / or from other environmental sensors of the driver assistance system 1. In this case, the size or extent of the respective obstacle Hi is determined. Depending on the size or contour, the data processing unit 4 calculates masking areas VB, which are respectively masked by the obstacles H and limit the field of view of an optical environmental sensor, for example a vehicle camera 2-i of the driver assistance system 1. In the Figure 1In the example shown, the light beam that impinges on the extreme contour point P1 of the obstacle H1 defines the occlusion area VB1, which limits the field of view FOV1 of the front vehicle camera 2-1. Similarly, the light beam that passes the extreme contour point P2 of the obstacle H2 defines the occlusion area VB2. This second occlusion area VB2 is limited on the one hand by the beam through point P2 and on the other hand by the outer line of the field of view FOV1 of the front vehicle camera 2-1. Furthermore, the obstacle H2 creates a further occlusion area VB3, which limits the field of view FOV2 of the left vehicle camera 2-2.

[0030] In one possible embodiment, the data processing unit 4 of the driver assistance system 1 processes image sensor data or camera images originating from optical environmental sensors 2-i in the determined obscuration areas VB. In one possible embodiment, the image sensor data or camera images are filtered in the determined obscuration areas VB. In another possible embodiment, the obscuration areas VB caused by the detected obstacles H are covered with texture surfaces or textures.

[0031] The driver assistance system 1 has a display on which the vehicle surroundings of the vehicle F can be shown to the driver of the vehicle F. For this purpose, camera images are projected onto a two-dimensional base surface or onto a bowl-shaped three-dimensional projection surface. In one possible embodiment, the data processing unit 4 calculates the obscuration areas VB caused by the detected obstacles H depending on this projection surface used for image display.

[0032] Figure 2shows a block diagram illustrating an embodiment of the driver assistance system 1 according to the invention. The data processing unit 4 evaluates the sensor data that it receives from environmental sensors in order to detect obstacles in the vehicle environment of the vehicle F. In this case, the environmental sensors comprise, in addition to the optical environmental sensors 2-i, in one possible embodiment, further environmental sensors with which sensor data for detecting obstacles H in the vehicle environment FU of the vehicle F are evaluated by the data processing unit 4. In the Figure 2 The block diagram shown shows an example of a further environmental sensor 5, which supplies sensor data for detecting obstacles H in the vehicle environment FU of the vehicle F. Furthermore, the driver assistance system 1 in the Figure 2illustrated embodiment, a control circuit 6 which, depending on the determined masking areas VB, optical environmental sensors, in particular those in Figure 2 illustrated vehicle cameras 2-1 to 2-4. In one possible embodiment, the control unit 6 switches between the image data streams generated by the vehicle cameras 2-i depending on the determined masking areas VB. In this case, the image sensor data or camera images are preferably passed through to the data processing unit 4, which have as few or as small masking areas VB as possible. Figure 1In the example shown, the front right area is normally imaged by the front camera 2-1 and the corresponding image data is transmitted to the data processing unit 4. Upon detection of the obstacle H1 and the resulting obscuration area VB1, the control unit 6 of the driver assistance system 1 can switch to the image data of the right vehicle camera 2-3, since the obstacle H1 does not cause an obscuration area VB within the field of view FOV3 of the right vehicle camera 2-3. Therefore, in the Figure 1In the scenario shown, the right vehicle camera 2-3 provides better quality image data than the front vehicle camera 2-1, in whose field of view FOV1 the obstacle H1 is located and causes an obscuration area VB1 there. When calculating the obscuration areas VB, in one possible embodiment, the data processing unit 4 takes into account the predetermined fields of view FOV of the relevant optical environmental sensors or vehicle cameras 2-i. In one possible embodiment, the fields of view or viewing angles of the vehicle cameras 2-i are stored in a configuration memory, which a processor of the data processing unit 4 has access to in order to calculate the obscuration areas VB depending on the read fields of view.

[0033] Figure 3 shows a further traffic scenario to explain the functioning of the driver assistance system 1 according to the invention. Figure 3In the traffic situation depicted, a vehicle F is moving along a road at a speed VF, with another vehicle, which represents an obstacle H1, approaching the vehicle F on the road. Next to the road to the right is an obstacle H2, for example within the field of view FOV of the front vehicle camera 2 of the vehicle F. The obstacle H2 creates an obscuration area VB2 within the field of view FOV of the vehicle camera 2, with the obscuration area VB2 changing depending on the driving speed VF of the vehicle F. In contrast to the fixed obstacle H2, for example a building, the other obstacle H1, namely the oncoming vehicle, is itself moving relative to the vehicle F.The vehicle H1 conceals an obscuration area VB1 within the field of view FOV of the front vehicle camera 2, wherein the obscuration area VB1 depends on the relative speed between the vehicle F and the vehicle H1. The data processing unit 4 of the driver assistance system 1 calculates the obscuration areas VB1, VB2 caused by the detected obstacles H1, H2 as a function of the relative speed between the detected obstacles and the vehicle F. For a fixed obstacle, such as the obstacle H2, the relative speed used is the vehicle's own speed VF.In the case of a mobile obstacle H1, for example an oncoming vehicle, the relative speed between the two vehicles is first determined based on the sensor data and then the obscuration area VB is calculated by the data processing unit 4 as a function of the determined relative speed. In one possible embodiment, the data processing unit 4 approximately calculates the area of ​​the respective obscuration area VB2. If, for example, the two surface contour points P1A, P1B of the oncoming obstacle H1 are far apart, the area of ​​the area VB1 obscured thereby is significantly larger than if the distance between the two contour points P1A, P1B is small. If the oncoming vehicle H1 is a truck, for example, the obscured area VB1 is significantly larger than for an oncoming car.The larger the area of ​​the occlusion area VB within the field of view FOV of the relevant camera 2, the greater the impairment of the image quality of the vehicle images supplied by the relevant vehicle camera. In one possible embodiment, the control unit 6 of the driver assistance system 1 also takes into account the size or proportion of the occlusion areas VB present in the field of view FOV of the relevant camera by switching or weighting the various camera image streams supplied by different cameras. For example, if the proportion of the sum of the occlusion areas VB (VB1 + VB2) in the camera images of the first front vehicle camera 2-1 is almost 50%, as in the traffic scenario according to. Figure 3 shown, from a certain threshold value onwards, the system switches, as far as possible, to camera images supplied by other vehicle cameras of vehicle F.

[0034] Figure 4shows a flowchart illustrating an embodiment of the method according to the invention for determining masking areas VB in the vehicle environment of a vehicle F.

[0035] In a first step S1, sensor data generated by environmental sensors of the vehicle F are evaluated to detect obstacles H in the vehicle environment of the vehicle F. This is done, for example, by a processor or microprocessor of the data processing unit 4 of a driver assistance system 1.

[0036] In a second step S2, occlusion areas VB or occlusion surfaces are calculated depending on the detected obstacles H. These occlusion areas VB are caused by obstacles H in the vehicle environment FU and restrict a field of view FOV of optical environmental sensors of the driver assistance system 1.

[0037] Figure 5shows a further exemplary embodiment of the method according to the invention. In one possible embodiment, after the occlusion areas VB have been determined, the image sensor data originating from the optical environmental sensors in which the determined occlusion areas VB are located are processed by the data processing unit 4 or another unit. In this case, the image sensor data of the relevant optical environmental sensors or vehicle cameras in whose field of view or viewing area the occlusion areas VB are located can be filtered. Alternatively, the determined occlusion areas within the camera images can be covered with textured surfaces.

[0038] In another possible embodiment, in step S3, the optical environmental sensors are controlled depending on the occlusion areas VB determined or calculated in step S2, for example, by switching between different environmental sensors. The size of the determined occlusion areas VB or their proportion of the total field of view FOV of the vehicle camera can be taken into account.

[0039] In one possible embodiment, in step S2, an obscuration area VB caused by a detected obstacle H is calculated as a function of the preconfigured field of view FOV of the relevant optical environmental sensor 2-i. Furthermore, in step S3, the obscuration areas VB are additionally calculated as a function of a relative speed between the detected obstacle H and the vehicle F.

[0040] The driver assistance system 1 according to the invention can be used for any vehicle, in particular road vehicles.

[0041] In one possible embodiment, the obscuration areas VB determined by the method according to the invention are evaluated for further functions of the driver assistance system 1. For example, in one possible application, a traffic density on a road can be derived from the proportion of the obscuration areas VB in the fields of view FOV of the vehicle cameras 2-i. For example, if a vehicle F is moving in the middle lane on a three-lane highway, large proportions of the fields of view FOV of the vehicle cameras 2 will be covered by other vehicles traveling in the same direction on the highway at high traffic density. The higher the traffic density on the road, the greater the probability of a traffic jam occurring, particularly if a traffic bottleneck occurs as a result of a vehicle accident on the road.The proportion of the obscuration areas VB within the field of view FOV of a camera 2 thus represents a measure of the current traffic density on the relevant road. In one possible embodiment, this traffic density measure can be evaluated for further functions of the driver assistance system 1.

Claims

1. Driver assistance system (1) for a vehicle (F), comprising: - environment sensors (2), which sense a vehicle environment of the vehicle (F), and comprising - a data processing unit (4), which evaluates sensor data of the environment sensors (2; 5) in order to detect obstacles (H) in the vehicle environment of the vehicle (F), - wherein, dependent on obstacles (H) detected, concealed regions (VB) in the vehicle environment of the vehicle (F) that are concealed by the obstacles (H) and restrict a field of view (FOV) of optical environment sensors (2) of the driver assistance system (1) are determined, - wherein the data processing unit (4) calculates the concealed regions (VB) caused by the detected obstacles (H) dependent on a relative speed between the detected obstacles (H) and the vehicle (F), - wherein a control circuit (6), which, dependent on the concealed regions (VB) caused by the detected obstacles (H), actuates optical environment sensors (2) and switches between different optical environment sensors (2) of the vehicle (F), is provided.

2. Driver assistance system according to Claim 1, wherein the data processing unit (4) processes, in particular filters, image sensor data that originate from optical environment sensors (2) in the determined concealed regions (VB).

3. Driver assistance system according to Claim 1, wherein the data processing unit (4) covers the concealed regions (VB) caused by the detected obstacles (H) with textured areas.

4. Driver assistance system according to one of the preceding Claims 1 to 3, wherein the data processing unit (4) calculates the concealed regions (VB) caused by the detected obstacles (H) dependent on a projection surface used for displaying images.

5. Driver assistance system according to one of the preceding Claims 1 to 4, wherein the optical environment sensors (2) have vehicle cameras, in particular fisheye cameras, which each have a predefined field of view (FOV).

6. Driver assistance system according to Claim 5, wherein the data processing unit (4) calculates the concealed regions (VB) caused by the detected obstacles (H) dependent on the fields of view (FOV) of the relevant optical environment sensors (2).

7. Method of determining concealed regions (VB) in the vehicle environment of a vehicle (F), comprising the following steps: (a) evaluating (S1) sensor data that are generated by environment sensors of the vehicle (F) in order to detect obstacles (H) in the vehicle environment of the vehicle (F); (b) calculating (S2), dependent on the detected obstacles (H), concealed regions (VB) that are concealed by the detected obstacles (H) and restrict a field of view (FOV) of optical environment sensors of the vehicle (F), wherein the concealed regions (VB) caused by the detected obstacles (H) are calculated dependent on a relative speed between the detected obstacles (H) and the vehicle (F); and (c) actuating (S3) the optical environment sensors (2) dependent on the concealed regions (VB) caused by the detected obstacles (H), by switching between different optical environment sensors (2) of the vehicle (F).

8. Method according to Claim 7, wherein sensor data that originate from optical environment sensors (2) of the vehicle (F) are processed, in particular filtered, in the calculated concealed regions (VB).

9. Method according to Claim 7, wherein the concealed regions (VB) caused by the detected obstacles (H) are covered with textured areas.

10. Method according to one of the preceding Claims 7 to 9, wherein the concealed regions (VB) caused by the detected obstacles (H) are calculated dependent on a projection surface used for displaying images.

11. Method according to one of the preceding Claims 7 to 10, wherein the concealed regions (VB) caused by the detected obstacles (H) are calculated dependent on predefined fields of view (FOV) of the relevant optical environment sensors (2).