Method for identifying a body shadow of a motor vehicle, driver assistance system for a motor vehicle, and motor vehicle

Abstract

The invention relates to a method (100) for recognizing a body shadow (10) of a motor vehicle (200), comprising the steps: determining a geographical position and an orientation of the motor vehicle (200); detecting at least one environment image (17a, 17b) by means of at least one camera (12) of a camera assembly (11) of the motor vehicle (200); determining a sun state with respect to the geographical position; determining from the at least one environment image (17a, 17b) by means of an evaluation device (13) whether a body shadow (10) of the motor vehicle (200) is to be expected; calculating a predicted body shadow (25) of the motor vehicle (200) by means of the evaluation device (13) with application of the sun state, the geographical position and the orientation of the motor vehicle (200) and information about the geometry of the motor vehicle (200); determining a body shadow candidate (28) in the at least one environment image (17a, 17b) by means of the evaluation device (13) with application of the predicted body shadow (25) if the body shadow (10) is to be expected; verifying the body shadow candidate (28) to be the body shadow (10) of the motor vehicle (200) by means of the evaluation device (13) by comparing the body shadow candidate (28) with the predicted body shadow (25).

Description

Technical Field

[0001] The present invention relates to a method for identifying the body shadow of a motor vehicle, comprising the steps of: determining the geographical location and orientation of the motor vehicle; detecting at least one environmental image by means of at least one camera of a camera assembly of the motor vehicle; determining the solar state with respect to the geographical location; and calculating a predicted body shadow of the motor vehicle by means of an evaluation device, applying information on the solar state, the geographical location and orientation of the motor vehicle, and the geometry of the motor vehicle.

[0002] Furthermore, the present invention also relates to a driver assistance system for a motor vehicle and a motor vehicle including the driver assistance system. Background Technology

[0003] For many camera-based driver assistance systems, the shadow cast by a vehicle presents a problem because it moves with the vehicle and changes according to its direction of travel. For example, in parking assistance systems, within the realm of so-called top-view visualization, artifacts can occur when the top-view visualization is composed of multiple camera images. This stitching effect then leads to unnatural shadow casting. Typically, false detections of the vehicle's body shadow often occur in functions utilizing camera image history. Another problem is that body shadows, due to their motion, can be confused with moving objects, and driver assistance systems may incorrectly react to this, for example, by applying full-speed braking. This problem arises especially under strong solar radiation and the associated constant, sharp shadow casting.

[0004] A method for operating a camera assembly with at least one camera for a vehicle is known from document DE 10 2011 122 454 A1, wherein the position of at least one ambient light source (whose light is incident on the vehicle) relative to the vehicle is determined, wherein at least one image is detected by means of the at least one camera, wherein the at least one image is evaluated based on the presence of at least one shadow by means of an evaluation device of the camera assembly, wherein if at least one shadow exists in the image, it is determined whether the at least one shadow is caused by light incident on the vehicle, wherein the at least one shadow is compared with reference information by means of the evaluation device, wherein the position of the at least one ambient light source relative to the vehicle is determined based on the at least one shadow.

[0005] Document JP 2011-065442 A discloses a vehicle shadow recognition device, which includes: a vehicle environment imaging device that images an area including a road surface (on which the user's vehicle is traveling); a sun position calculation device that calculates the sun position obtained by the device and information about the current date from information about the vehicle's current position; a vehicle shadow region estimation device that estimates the area occluded by the vehicle based on information about the vehicle's direction of travel and information about the vehicle's three-dimensional shape; and a vehicle shadow region detection device that identifies the vehicle shadow region in the image obtained by the vehicle environment imaging device based on coordinate information obtained from the vehicle shadow region estimation device.

[0006] The source of error in methods for identifying the body shadow of a motor vehicle is an incorrect determination of the current solar condition or the sun being obscured by clouds or buildings. In such cases, the body shadow may not be correctly identified, or a shadow not caused by the vehicle may be identified as the body shadow. Summary of the Invention

[0007] The purpose of this invention is to provide a method for identifying the body shadow of a motor vehicle, thereby improving the identification of the body shadow of a motor vehicle.

[0008] To achieve the objective of this invention, a method for identifying the body shadow of a motor vehicle is proposed, comprising the following steps: - Determine the geographical location and orientation of motor vehicles. - Detect at least one environmental image using at least one camera from a camera assembly in a motor vehicle. - Regarding the determination of the sun's state by geographical location - Using an evaluation device, determine whether the vehicle's body shadow should be expected from the at least one environmental image. - The predicted body shadow of a vehicle is calculated using an evaluation device, taking into account the solar state, the vehicle's geographic location and orientation, and information about the vehicle's geometry. - If a body shadow is expected, body shadow candidates are determined in at least one environmental image using an evaluation device with the predicted body shadow applied. - The candidate body shadow is verified to be the body shadow of a motor vehicle by comparing it with the predicted body shadow using an evaluation device.

[0009] Body shadow, also known as self-shadow, refers to the shadow cast by a vehicle in a motor vehicle environment due to sunlight.

[0010] The motor vehicle used to perform this method includes a camera assembly with at least one camera. Such cameras are installed in known motor vehicles for various vehicle functions. Thus, for driver assistance systems, camera assemblies are known for evaluating traffic conditions. Furthermore, modern motor vehicles include camera assemblies with cameras for generating so-called top-view diagrams of the motor vehicle, for example, for parking assistance systems.

[0011] According to this method, at least one environmental image is detected using at least one camera of the camera assembly. However, it is also possible to detect multiple environmental images using multiple cameras of the camera assembly.

[0012] The evaluation device may include one or more electronic computing devices and, in particular, may have a CPU and memory. Alternatively, the evaluation device may be an evaluation device for a motor vehicle and disposed therein. Furthermore, it is possible that the evaluation device is an external evaluation device that can exchange data with the motor vehicle's computing unit, for example, via a mobile communication connection.

[0013] The geometry of the motor vehicle is preferably stored in the memory of the evaluation device and the evaluation device retrieves the ontological shadow from the memory for calculating the prediction.

[0014] According to the invention, an evaluation device is used to determine whether a vehicle body shadow should be expected from the at least one environmental image. Situations where a vehicle body shadow should not be expected include strong cloud cover or the sun being blocked by buildings, other vehicles, or infrastructure in the vehicle's environment.

[0015] According to the present invention, it is further configured such that the determination of a body shadow candidate in the environmental image is preferably performed only if a body shadow should be expected. In other words, if a vehicle's body shadow should not be expected, a body shadow candidate is not determined in the environmental image. This avoids misdetecting shadows not caused by vehicles as body shadows. Furthermore, it can be configured such that the step of calculating the predicted body shadow is performed only if a vehicle's body shadow should have been determined.

[0016] Preferably, in order to determine whether a body shadow should be expected, an exposure comparison is performed between multiple regions of the at least one environmental image and / or between multiple environmental images.

[0017] If the environmental image detected by at least one camera of the camera assembly has areas with extremely different exposures, or if the environmental image detected by multiple cameras of the vehicle's camera assembly has extremely different exposures or areas with extremely different exposures—that is, if an area or environmental image is overexposed compared to other areas or other environmental images—it can be inferred that the sun is shining or that the sun is not obscured by clouds, buildings, or vehicles. This can then be assessed as the expected shadow cast by the vehicle itself.

[0018] If an exposure comparison between different areas of the at least one environmental image or between multiple environmental images reveals a constant maximum exposure, this indicates that direct solar radiation is blocked by clouds or surrounding buildings, and therefore, the vehicle's body shadow should not be expected. In this case, it is not determined whether the shadow measured from the environmental image is the vehicle's body shadow, thus avoiding the misdetection of shadows not caused by the vehicle as the vehicle's body shadow.

[0019] Preferably, if it is determined from the at least one environmental image that there is no object in the line of sight between the sun and the geographical location of the vehicle, a body shadow should be expected.

[0020] It can also be configured such that if an object is determined from the at least one environmental image to be present in the line of sight between the sun and the vehicle's geographical location, a body shadow should not be expected.

[0021] For example, image recognition methods can be used to determine whether the sun can be identified in a segment of an environmental image. If the sun is identified in the environmental image, there is a free line of sight between the sun and the vehicle's geographical location, and therefore the vehicle's shadow should be expected.

[0022] If the sun cannot be determined in the environmental image or multiple environmental images, this indicates the presence of an object in the line of sight between the sun and the vehicle's geographical location. In this case, a body shadow that should not be expected can be determined.

[0023] Furthermore, it can be advantageously set that the object is a cloud, vehicle, tree, or building.

[0024] It can be further advantageously configured that the solar status includes altitude and horizontal angle.

[0025] The horizontal angle is also called the azimuth angle.

[0026] It is further advantageous to set up a system that determines the height and / or horizontal angle from the geographical location of the motor vehicle and / or the orientation of the motor vehicle and / or the date and / or time.

[0027] Knowing the vehicle's geographical location and / or orientation, as well as the date and / or time, the solar state, especially the solar altitude and / or horizontal angle, at the vehicle's geographical location can be determined using known mathematical methods or by evaluating a solar state table.

[0028] It can be further advantageously configured that the height and / or horizontal angle are determined from the at least one environmental image.

[0029] Typically, the sun's altitude can be well determined from the geographical location of the vehicle being measured, as well as the date and time of application. However, if the vehicle's orientation has significant uncertainty, the horizontal angle may only be determined with limited precision. In such cases, it is particularly advantageous to assess environmental images using evaluation equipment to determine the sun's horizontal angle and / or altitude.

[0030] For example, if the sun is identified in at least one environmental image using image recognition methods, the height and / or horizontal angle can be determined by combining this information with the location of the camera detecting the environmental image at the vehicle. As mentioned above, the expected shadow of the vehicle body can also be determined in this case.

[0031] Preferably, the predicted body shadow is the projection of the vehicle's geometry onto a plane and / or onto the ground plane.

[0032] Therefore, the predicted body shadow is calculated using known mathematical methods by projecting the geometry of the vehicle, preferably stored in the memory of the vehicle's evaluation device, onto a plane. Here, this plane is preferably determined to be approximately parallel to the ground plane where the vehicle is located. In principle, it is also possible that the predicted shadow is calculated as a projection onto that ground plane corresponding to the actual situation, given an understanding of the geometry of the ground (e.g., the road surface) where the vehicle is located. Distortions in the body shadow due to the geometry of the ground plane can thus be considered.

[0033] It is further advantageously configured that a first line segment of a preferred straight line is determined to represent the edge of the predicted body shadow, a second line segment of the preferred straight line is determined in the environmental image when the first line segment is applied, and the second line segment is associated with the edge of the body shadow candidate.

[0034] Therefore, the predicted body shadow can be used as a starting approximation for determining body shadow candidates. Line segments can be identified in the at least one environmental image using image recognition methods. If these line segments have the same or similar location and / or orientation as the first line segment, the line segments identified in the environmental image are determined as second line segments representing the edges of the body shadow candidates. The first line segment is preferably used as a first approximation to limit the search area for the second line segment. This approach is advantageous because the straight or only slightly curved edges of the shadow (i.e., the boundary between the shadow and the surrounding unmasked area) can be identified more easily and reliably using image recognition methods. Furthermore, the computational cost for determining body shadow candidates in the environmental image is reduced.

[0035] The first line segment may also be slightly rounded and preferably corresponds to the generally straight profile of the motor vehicle, such as the roof profile or engine hood.

[0036] After determining the candidate body shadow, it is verified whether the candidate body shadow is indeed the body shadow of a motor vehicle. To do this, a comparison is performed between the position and / or orientation of the first line segment of the predicted body shadow and the second line segment of the candidate body shadow.

[0037] Alternatively, the second line segment can be connected to form an outline. This will give you the complete edge of the body shadow candidate.

[0038] The continuous edges of the body shadow candidate are approximated by connecting the second line segment. If the body shadow candidate is the body shadow of a motor vehicle, this approximation of the body shadow candidate's edges allows for a more accurate comparison with the predicted body shadow.

[0039] This can be configured to connect the second line segments based on the brightness gradient of the at least one environmental image, and / or based on the extrapolated intersections of the second line segments.

[0040] The connection of the second line segment can also be made using artificial intelligence methods (such as learned neural networks).

[0041] Preferably, the location of the vehicle is determined by means of data from a satellite navigation system, and / or the geographical orientation of the vehicle is determined by means of a rotation rate sensor and / or a compass.

[0042] Another implementation of the objective of this invention is to provide a driver assistance system for a motor vehicle, which is configured to perform the above-described method.

[0043] If the aforementioned method is implemented during the operation of a motor vehicle driver assistance system, false detections of the vehicle's body shadow can be reduced.

[0044] Preferably, the driver assistance system is an emergency braking assist system or an automatic driving system. Preferably, the measured body shadow can be discarded for evaluating traffic conditions.

[0045] If the driver assistance system is an emergency braking assist system or an automatic driving system, the risk of the body shadow being confused with moving objects due to its motion is reduced. This can help avoid dangerous traffic situations while the driver assistance system is in operation.

[0046] Furthermore, it can be advantageously configured such that the driver assistance system is an illustration of the vehicle from a bird's-eye view, and the measured body shadow is used to realistically represent the body shadow in the illustration of the vehicle.

[0047] A bird's-eye view of a vehicle can also be called a top-view illustration and is particularly used for visualization in parking assistance systems. The measured body shadow can be used to provide the true outline of the measured body shadow in a bird's-eye view illustration of a vehicle, or to remove the body shadow from the illustration. This avoids stitching effects that can occur due to stitching together images of different environments.

[0048] Another implementation of the objective of this invention is a motor vehicle that includes the aforementioned driver assistance system. Attached Figure Description

[0049] The invention will now be described in more detail with reference to the accompanying drawings. Wherein: Figure 1 A flowchart of a method for identifying the body shadow of a motor vehicle is shown. Figure 2 Motor vehicles were shown. Figure 3 The first environmental image is shown. Figure 4 A second environmental image is shown. Figure 5 The vehicle shown has a free line of sight to the sun. Figure 6 The image shows a motor vehicle with its view of the sun obstructed. Figure 7 A motor vehicle with both body shadow and predicted body shadow is shown, as well as Figure 8 A motor vehicle is shown with a body shadow and a predicted body shadow with the determined line segments.

[0050] In the accompanying drawings, the same or corresponding parts are indicated by the same reference numerals. Detailed Implementation

[0051] Figure 1A flowchart of a method 100 for identifying the body shadow 10 of a motor vehicle 200 is shown. For further explanation of method 100, see also another... Figures 2 to 8 .

[0052] Figure 2 A motor vehicle 200 configured to perform method 100 is shown. The motor vehicle 200 includes a camera assembly 11 with multiple cameras 12. Furthermore, the motor vehicle 200 includes an evaluation device 13. Software or a computer program is implemented on the evaluation device 13 for performing method 100 for identifying body shadows 10. The motor vehicle 200 also includes a receiver 14 for a satellite navigation system and a rotation rate sensor or compass 15. Additionally, information about the geometry of the motor vehicle 200 is stored in the memory 16 of the evaluation device 13.

[0053] According to Figure 1 In the first method step S1 of the flowchart, the geographical location of the vehicle 200 is determined by means of a receiver 14 for a satellite navigation system. Furthermore, the orientation of the vehicle 200 is determined based on data from a compass 15 or a rotation rate sensor. In the second method step S2, environmental images 17a, 17b are detected by means of camera 12 of the camera assembly 11 of the vehicle 200. Figure 3 and Figure 4 Such environmental images 17a and 17b are shown.

[0054] In the third method step S3, the solar state is determined regarding the geographical location. According to... Figure 5 The solar state includes altitude 18 and horizontal angle 19, also known as azimuth. To determine the solar state, the geographical location, date, and time determined by means of receiver 14 for a satellite navigation system can be used to determine the solar state applicable to the geographical location, for example, from a table or by means of known calculation methods. Alternatively or additionally, the solar state can also be determined by means of at least one 17b in an environmental image. Figure 4 The altitude 18 and horizontal angle 19 of the sun are determined by directly identifying the sun 20 in the evaluation device 13. For this purpose, an image recognition method performed on the evaluation device 13 can be used.

[0055] In the fourth method step S4, the potential for a body shadow 10 of the vehicle 200 is determined from the environmental images 17a, 17b by means of an image recognition method performed on the evaluation device 13. This can be performed, for example, by means of exposure comparison between multiple regions 21a, 21b, 21c of the environmental images 17a, 17b. If a strong exposure difference can be identified between regions 21a, 21b, 21c, the potential for a body shadow 10 of the vehicle 200 can be inferred. Alternatively or additionally, the potential for a body shadow 10 of the vehicle 200 can also be determined if the sun 20 is directly identified in at least one of the environmental images 17b.

[0056] If the shadow 10 of the vehicle body 200 is expected, for example because the sun 20 is identified in an environmental image 17b, then as in Figure 5 The image shows a free line of sight 22 between the sun 20 and the vehicle 200. If, from the at least one environmental image 17a, 17b, an object 23, such as a building 24, is determined to exist within the line of sight 22 between the sun 20 and the vehicle 200, then... Figure 6 As shown, the shadow 10 of the motor vehicle 200 should not be considered.

[0057] In the fifth method step S5, the predicted body shadow 25 of the vehicle 200 is calculated by means of the evaluation device 13, taking into account the solar state, the geographical location and orientation of the vehicle 200, and information about the geometry of the vehicle 200. Known mathematical methods are used for this purpose. The predicted body shadow 25 is calculated here as a projection of the geometry of the vehicle 200 onto plane 26 (which corresponds to the ground plane 27).

[0058] In the sixth method step S6, if a body shadow 10 is expected, a body shadow candidate 28 is determined in the at least one environmental image 17a by means of the evaluation device 13 with the application of the predicted body shadow 25. For this purpose, as in Figure 7 and Figure 8 As shown, a first line segment 29 of the straight line that forms the edge 30 of the predicted body shadow 25 is determined. Line segments are searched in the at least one environmental image 17a using an image recognition method. If these line segments have the same or similar position and / or orientation as the first line segment 29, they are identified as second line segments 31 of the edge 32 of the body shadow candidate 28. The first line segment 29 is thus used as a first approximation to limit the search area for the second line segment 31.

[0059] Then, in the seventh method step S7, the positions and / or orientations of the first line segment 29 and the second line segment 31 are compared. If the first line segment 29 can be associated with the second line segment 31 accordingly, the candidate body shadow 28 determined from the environmental image 17a is the body shadow 10 of the motor vehicle 200.

[0060] return Figure 2 The motor vehicle 200 also includes a driver assistance system 300, which is configured as an emergency braking assist system 33 and processes information about the body shadow 10 measured by means of method 100.

[0061] List of reference numerals 100 methods 200 motor vehicles 300 Driver Assistance System 10. Body Shadow 11 Camera Components 12 cameras 13. Evaluation equipment 14 Receivers 15. Compass 16. Memory 17a Environmental Images 17b Environmental Images 18 Height 19 Horizontal Angle 20 Suns Area 21a Area 21b 21c area 22 lines of sight 23 objects 24 buildings 25. Predicted body shadow 26 Planes 27. Ground level 28 Candidates for Body Shadow 29 First line segment 30 Edge 31 Second line segment 32 Edge 33 Emergency Braking Assist System S1-S7 Method Steps.

Claims

1. A method (100) for identifying the body shadow (10) of a motor vehicle (200), comprising the steps of: - Determine the geographical location and orientation of the motor vehicle (200). - At least one environmental image (17a, 17b) is detected by means of at least one camera (12) of the camera assembly (11) of the motor vehicle (200). - Regarding the determination of the solar state at the aforementioned geographical location. - Using an evaluation device (13), determine whether the body shadow (10) of the motor vehicle (200) should be expected from the at least one environmental image (17a, 17b). - Using the evaluation device (13), the predicted bulk shadow (25) of the motor vehicle (200) is calculated by applying the solar state, the geographical location and orientation of the motor vehicle (200), and information about the geometry of the motor vehicle (200). - If a body shadow (10) is expected, a body shadow candidate (28) is determined in the at least one environmental image (17a, 17b) by means of the evaluation device (13) with the application of the predicted body shadow (25). - The body shadow candidate (28) is verified to be the body shadow (10) of the motor vehicle (200) by comparing the body shadow candidate (28) with the predicted body shadow (25) using the evaluation device (13).

2. The method (100) according to claim 1, wherein, To determine whether a body shadow (10) should be expected, an exposure comparison is performed between multiple regions (21a, 21b, 21c) of the at least one environmental image (17a, 17b) and / or between multiple environmental images (17a, 17b), and / or, wherein if it is determined from the at least one environmental image (17a, 17b) that no object (23) is located on the line of sight (22) between the sun (20) and the geographical location of the vehicle (200), a body shadow (10) should be expected, and / or, wherein if it is determined from the at least one environmental image (17a, 17b) that the object (23) is located on the line of sight (22) between the sun (20) and the geographical location of the vehicle (200), a body shadow (10) should not be expected.

3. The method (100) according to claim 2, characterized in that, The object (23) is a cloud, vehicle, tree, or building (24).

4. The method (100) according to any one of the preceding claims, characterized in that, The solar state includes an altitude (18) and a horizontal angle (19), wherein, preferably, the altitude (18) and / or the horizontal angle (19) are determined from the geographic location of the vehicle (200) and / or the orientation of the vehicle (200) and / or the date and / or the time, and / or, wherein, preferably, the altitude (18) and / or the horizontal angle (19) are determined from the at least one environmental image (17a, 17b).

5. The method (100) according to any one of the preceding claims, characterized in that, The predicted body shadow (25) is the projection of the geometry of the vehicle (200) onto the plane (26) and / or onto the ground plane (27).

6. The method (100) according to any one of the preceding claims, characterized in that, The first line segment (29) of the preferred straight line of the edge (30) of the predicted body shadow (25) is determined, and the second line segment (31) of the preferred straight line is determined in the environmental image (17a, 17b) when the first line segment (29) is applied, and the second line segment (31) is associated with the edge (32) of the body shadow candidate (28).

7. The method (100) according to claim 6, characterized in that, The second line segment (31) is connected to form a contour, wherein, preferably, the connection of the second line segment (31) is based on the brightness gradient of the at least one environmental image (17a, 17b), and / or, wherein, preferably, the connection of the second line segment (31) is based on the extrapolated intersection of the second line segment (31).

8. The method (100) according to any one of the preceding claims, characterized in that, The geographical location of the motor vehicle (200) is determined by means of data from a satellite navigation system, and / or the geographical orientation of the motor vehicle (200) is determined by means of a rotation rate sensor and / or a compass (15).

9. A driver assistance system (300) for a motor vehicle (200), configured to perform the method (100) according to any one of the preceding claims.

10. A motor vehicle (200) comprising a driver assistance system (300) according to claim 9.

Images