Device for the verified detection of a road user and device for a vehicle for the verified detection of a road user

Abstract

Method for the verified detection of a road user based on the detection of at least one light source in an image of a vehicle's surroundings, the method further comprising the steps Providing at least one initial image of the vehicle's surroundings (S1), Detecting an object in the vicinity of the vehicle based on the detection of a light source of the object in the first image (S2), Verify whether the detected object is a road user, whereby verification is carried out by means of a distance calculation (S3), Determining an initial distance value between the vehicle and the detected object based on the first image, Determining a second distance value of the vehicle to the detected object based on a second image of the vehicle's surroundings, and where the verification of the object detection is carried out by comparing the first and second distance values.

Description

Technical field

[0001] The present invention relates to light assistance systems. In particular, the present invention relates to a method for the verified detection of a road user based on the detection of at least one light source in an image of a vehicle's surroundings and to a device for a vehicle for the verified detection of a road user. Technical background

[0002] Lighting functions in camera-based driver assistance systems are currently implemented using a monocular camera as the primary sensor. Various types of lighting assistants are used for this purpose in motor vehicles. For example, well-known systems such as High Beam Assist, Intelligent Headline Assist, and Glare-Free High Beam are considered state-of-the-art in vehicles.

[0003] One challenge with the lighting function is reliably distinguishing between vehicles and other self-illuminating objects, such as reflectors, traffic lights, or road signs. A reliable technique for detecting other road users, especially vehicles ahead, is the pairing of symmetrical lights at the same height, as this is almost exclusively found on cars. However, the problem here is that more and more LED signs are appearing on multi-lane roads, often displaying the same traffic sign in pairs. Because these signs have a high red component, look identical, and are at the same height, they are usually automatically paired by the driver assistance system or the light assistant, leading to a false positive detection. Specifically, this means that two LED signs at a distance appear to the camera as a single vehicle approaching at close range. Summary of the invention

[0004] It can be considered an object of the invention to provide an improved control of the headlights in vehicles. Likewise, it can be considered an object of the invention to provide an improved detection of vehicles or other road users ahead.

[0005] The problem is solved by the subject matter of the independent claims. Further developments and additional embodiments are specified in the dependent claims, the following description, and the figures.

[0006] The described embodiments relate equally to the method for the verified detection of a road user, as well as to the device and the vehicle. In other words, features described below in relation to the method can also be implemented in the device or the vehicle and can be considered corresponding features or configurations of the device. The reverse is also true, of course. In particular, the device is designed to perform the methods described below, unless explicitly stated otherwise.

[0007] According to one embodiment of the invention, a method for the verified detection of a road user based on the detection of at least one light source in an image of a vehicle's surroundings is described. The method includes the step of providing at least one first image of the vehicle's surroundings. Detecting an object in the vehicle's surroundings based on the detection of a light source of the object in the first image is a further step in this embodiment of the method. Verifying whether the detected object is a road user, whereby the verification is performed by means of a distance calculation, is also part of this method.

[0008] Object detection can be achieved, for example, through light pairing. A pairing unit can be used to detect one or more light sources of an object in the first image. This allows the pairing unit to detect the object in the vicinity of the vehicle that possesses the light sources. Additionally, if desired, an initial distance can be calculated based on the information from the first image, for example, by means of an estimate. Details on examples of estimation methods are provided below. Independently of this, a second distance calculation can be performed, for example, using disparities detected in a first and second image from a stereo camera. Verification in this example is achieved by comparing the first and second distance values.Further aspects of this will be described below using various examples.

[0009] The verification step using distance calculation can, for example, involve a three-dimensional position calculation of the lights to be verified. Different images from a vehicle's stereo camera can be used to check whether the detected object containing the light sources is indeed another road user, particularly a vehicle ahead. In other words, this method allows for the identification of another vehicle. Verification can involve comparing two distance values ​​determined by different methods. This comparison confirms whether the vehicle in question is indeed ahead. In certain implementations, color recognition can also be performed. Details will be provided using various examples.

[0010] The method according to the invention therefore avoids false positive detection, which occurs as a disadvantage in the prior art described above. In particular, the method according to the invention does not detect, for example, reflectors, traffic lights, road signs, and especially LED signs, which are located at a distance and could appear as if they were vehicles ahead at a short distance, as other road users, especially not as vehicles ahead. This allows for improved control of the vehicle's front lighting, since no or at least fewer false detection events occur.

[0011] For example, this method can be implemented in a vehicle's light assistant, which performs a stereo distance calculation based on two different images from a stereo camera. This calculation verifies the detected object and its distance. As will be explained in more detail below, a preferred embodiment of the invention involves pairing at least two light sources contained in one image. However, it is also possible for the inventive method to verify the distance between individual red lights using stereo information. Further details will be explained below.

[0012] In particular, in the context of the present invention, the term "object" can be regarded as a potential vehicle, whereby verification is carried out to determine whether it is actually a vehicle or whether it is a false positive detection of, for example, a traffic light or an LED sign.

[0013] According to a further embodiment of the invention, a pairing of light sources contained in the first image is carried out, whereby the object containing the light sources is detected.

[0014] In particular, the device according to the invention can include a pairing unit designed to pair lights that are contained in the first image and that belong to the detected object. Specifically, pairs of light sources located at the same horizontal height can be formed. Pairing symmetrical lights arranged at the same height in the images is a reliable technique, as these occur almost exclusively with cars, i.e., other road users driving ahead. Verifying the method according to the invention based on an additional distance calculation of the detected object eliminates false positive detection.

[0015] According to a further embodiment of the invention, the distance calculation is performed as a stereo distance calculation based on a second image of the vehicle's surroundings. The first and second images are generated by a stereo camera mounted on the vehicle.

[0016] In this embodiment, a first distance value of the detected object to the vehicle can be determined based on the first image from the stereo camera. This first distance value can be determined, for example, by estimating the distance of the potential vehicle based on the average width of cars and the distance between the two lights in the first image from the stereo camera. A second distance value of the object to the vehicle can then be determined based on the second image from the stereo camera. In particular, the second calculation of the second distance value can be a stereo distance calculation. For example, a three-dimensional position calculation can be performed using the second image from the stereo camera of the vehicle, taking into account the disparities of the lights to be verified, so that the distance is obtained in the form of the second distance value of the paired lights to the vehicle itself.The two pictures may have been taken at the same time or at nearly the same time.

[0017] In this embodiment, both partial images from the stereo camera are used for verified detection of the road user. An exemplary device that uses a stereo camera is the one described below. Fig. 1 can be seen.

[0018] According to the invention, the method includes determining a first distance value of the vehicle to the detected object based on the first image. Determining a second distance value of the vehicle to the detected object based on a second image of the vehicle's surroundings is also part of the method. Verification of the object detection is then performed by comparing the first and second distance values.

[0019] The two distance values ​​can be determined by one and the same computing unit. However, it is also possible for different, structurally separate devices to perform the first and second determinations of the first and second distance values ​​separately.

[0020] According to a further embodiment of the invention, the second distance value is determined based on a different horizontal position of the light sources in the first and the second image.

[0021] The first and second images could, for example, come from a stereo camera on the vehicle. However, it is also possible that the first image is generated by one camera and the second image by a second camera.

[0022] According to a further embodiment of the invention, a three-dimensional position calculation of the detected object is carried out by means of disparities of the light sources with respect to the first and the second image.

[0023] The resulting three-dimensional position can be used to determine whether the previously detected object is indeed another road user, such as a vehicle ahead. If comparing the two distances reveals that the object is at a greater distance, for example, a distant LED sign, appropriate action can be taken. For instance, a previously established pairing of two lights can be reversed. Other measures are also possible.

[0024] Disparity, also known as lateral disparity in stereoscopy, refers to the spatial offset of the same object in two different images. Ideally, the two images should have been captured at the same time or nearly at the same time. This is part of an embodiment of the invention. Another embodiment of the invention involves calculating the distance from the cameras / stereo camera to the object / light source(s) by means of the offset of the object / light source(s) in the image horizontal plane, the horizontal distance between the two cameras or the horizontal distance between two sensors of a stereo camera, and the focal length of the cameras or the focal length of the stereo camera.

[0025] This embodiment, based on the use of disparities in a first mono image and a second mono image from a stereo camera, can be used with the pairing process described above. In particular, pairing can be performed in two stages in this embodiment, as explained in more detail below. In a first stage, symmetrical lights at the same height, located in the first image, are paired to form a light source pair. In a second stage, it can be checked whether the lights have a strong red component. This allows verification of whether the object could potentially be a vehicle ahead. Subsequently, the verification according to the invention is carried out. Using the second partial image from the mono camera, a three-dimensional position calculation is performed based on the disparities of the lights to be verified, thus obtaining the distance of the paired lights to the vehicle being tested.Using the distance between the two lights in the single image and the average width of vehicles, the distance of the potential vehicle is estimated, yielding an initial distance value. The second distance value in this example is obtained through a three-dimensional position calculation using disparities. If the result of the estimate (i.e., the first distance value) indicates a nearby vehicle, but the result of the disparity calculation is a distant object, then the detected object must be, for example, LED signs. However, it can be ruled out that it is a nearby vehicle. If the estimate indicates a nearby vehicle and the disparity calculation also indicates a nearby object, then it is indeed a vehicle.Subsequently, the vehicle's front lighting can be controlled based on the result of comparing the two distance values.

[0026] According to a further embodiment of the invention, the pairing can be corrected or confirmed using the result of the verification. For example, the verification phase described above can be performed only for red lights that are already paired, thus requiring few additional resources.

[0027] According to a further embodiment of the invention, the determination of the first distance value is carried out as an estimate of the first distance value by means of a distance between two light sources of the object in the first image and by means of an average width of motor vehicles.

[0028] According to a further embodiment of the invention, the vehicle's headlights are controlled based on the result of verifying whether the detected object is a road user.

[0029] In particular, the device can be controlled by a light assistant or a driver assistance system. Exemplary embodiments of such a device will be explained in more detail in the context of the figure descriptions.

[0030] According to a further embodiment of the invention, a device for a vehicle is specified, wherein the device is designed for the verified detection of a road user based on the detection of at least one light source. The device includes a processing unit, the processing unit being configured to verify the detection of an object in the vicinity of the vehicle depicted in the first image. The object detection is based on the detection of a light source of the object in the first image. Furthermore, the processing unit is configured to perform the verification by calculating the distance between the detected object and the vehicle.

[0031] For example, the device could be a light assistant with 3D correction. It could also be designed to control the vehicle's front lighting. The processing unit could be a processor, microcontroller, or computer, which may be located inside or outside the vehicle. In particular, the processing unit could be connected to other components of the device via wired or wireless connections. Specifically, the camera, especially a stereo camera, could be part of the device with which the processing unit communicates. The device could also include a pairing unit for identifying lights depicted in an image. In another embodiment, the device includes a color recognition unit for identifying red lights in an image.

[0032] In other words, the device for verifying the detection of a road user, in particular a vehicle ahead, is designed based on light signals within the first and a second image.

[0033] In one exemplary embodiment, the device is configured as a system within a vehicle to control the front lighting. A camera is integrated into the device, which is used to detect other road users based on their taillights and headlights. Additionally, symmetrical lights are paired within the device. Using the processing unit according to the invention, these paired lights are verified via a stereo distance calculation. This device can also be fully functional without a stereo image being provided. A stereo image is only used for verification purposes. Furthermore, the distance between individual red lights can also be verified using stereo information.

[0034] According to a further embodiment of the invention, a program is specified which, when executed on a processor, instructs the processor to carry out a method described in the context of the present invention. The program element can be part of a computer program. Furthermore, the program element can also be an independent computer program itself. For example, the program element can enable an existing computer program to execute the method according to the invention by acting as an update.

[0035] According to a further embodiment of the invention, a computer-readable medium is provided on which a program element is stored which, when executed on a processor, instructs the processor to carry out a method described in the context of the present invention. The computer-readable medium can be considered a storage medium, for example, a USB flash drive, a CD, a DVD, a hard drive, or another storage medium. Furthermore, the computer-readable medium can be implemented as a microchip that enables a driver assistance system, in particular a light assistant, in a vehicle to carry out the method according to the invention.

[0036] Further advantages, features, and applications of the invention will become apparent from the following description of the exemplary embodiments and figures. All features described and / or illustrated, individually and in any combination, constitute the subject matter of the invention, irrespective of their composition in the individual claims or their cross-references. Brief description of the characters Fig. Figure 1 shows a device for the verified detection of a road user based on the detection of at least one light source according to an embodiment of the invention. Fig. Figure 2 shows a vehicle with a device according to an embodiment of the invention. Fig. Figure 3 shows a flowchart of a process according to an embodiment of the invention.

[0037] The figures are shown schematically and not to scale. If the same or similar reference symbols are used in different figures in the following description, these denote the same or similar elements. Detailed description of implementation examples

[0038] Fig. Figure 1 shows a device 100 for a vehicle for the verified detection of a road user 201 based on the detection of light sources. The device 100 can, for example, be installed in the car 200 of the Fig. 2 for the detection of light sources 208, 209. The device 100 has a computing unit 101, which is configured to verify the detection of an object in the vicinity of the vehicle based on the detection of the object's light sources in the first figure 105. Furthermore, the computing unit 101 is configured to perform the verification by calculating the distance of the detected object to the vehicle. In the exemplary embodiment of the Fig. The device 100 is designed to control a front light 109 of the vehicle (not shown here). For example, the device 100 can be considered a light assistant or a driver assistance system that controls the light. In addition to the processing unit 101, the device 100 has a pairing unit 110. The pairing unit 110 is designed to detect one or more light sources of an object in the first image. Thus, the pairing unit 110 detects the object in the vicinity of the vehicle that has the light sources.

[0039] The pairing unit 110 receives the first image 105 from the first sensor 103 of the stereo camera 102. The color recognition unit 107 can determine whether red lights are present in the first image 105, which the device uses as a criterion to determine whether the detected object is a vehicle ahead. If the color recognition unit 107 detects red lights in the first image 105, thus answering the question "yes," the processing unit 101 can verify whether the detected object is a road user. The processing unit 101 performs this verification by calculating a distance. This distance calculation is based on the second image 106, which is provided to the processing unit 101 by the second sensor 104 of the stereo camera 102. The transmission of the first and second images can be wired or wireless.In particular, the stereo camera 102 can be part of the device 100, but it can also be arranged separately in the vehicle.

[0040] In other words, the device 100 can be considered a system used in a vehicle to control the front lighting. The stereo camera 102 detects other road users using images generated by the two sensors 103 and 104. Based on the taillights and headlights, the pairing unit 110 additionally performs pairing within the first image generated by the stereo camera 102. Pairing can only be performed if, for example, two light sources in the first image are positioned at the same height. Likewise, pairing can only occur if the two light sources are symmetrical, meaning that both light sources have an identical or very similar shape.

[0041] The verification according to the present invention is carried out in the exemplary embodiment of the Fig. 1 is only carried out if the color recognition unit 107 determines that the paired lights have a minimum red component.

[0042] For example, a value for a minimum red component can be stored in a memory unit of the device or even outside the device, which is used as a limit value. If the paired light sources in the first image do not have a minimum red component, component 108 can control the headlights 109 of the vehicle accordingly. If the paired lights have a strong red component and could therefore be a vehicle ahead, the verification according to the invention is carried out in the processing unit 101. Based on the result of the verification as to whether the detected object is indeed a road user, i.e., a vehicle ahead, the pairing can be corrected or confirmed. This correction or confirmation is in the Fig. 1 symbolically represented by the arrow 111.

[0043] According to another embodiment, however, it is also possible for the first computing unit 101 to directly control the vehicle lights 109. This is shown in Fig. 1 is symbolically represented by the arrow 112. The pairing unit 110 can estimate the distance of the potential vehicle based on the first monoscopic image 105 and an average vehicle width. This determines an initial distance value. Furthermore, the processing unit 101 can perform a three-dimensional position calculation based on the second partial image from the camera, using disparities in the lights to be verified. This yields a second distance value between the paired lights and the vehicle being tested. In this embodiment, the verification is performed by comparing the first and second distance values, as calculated by the processing unit 101. If the estimated distance is small and the disparity calculation is large, then the vehicle ahead is not a vehicle, i.e., not another road user.However, if the result of the estimation is a vehicle traveling nearby and the result of the calculation using disparities is also a nearby object, then it is indeed a vehicle. Based on these two different possible results, the device 100 can control the vehicle lights 109.

[0044] The first processing unit 101 can determine the disparity, i.e., the spatial offset of the light sources in the two images 105 and 106. The two images 105 and 106 should ideally have been captured at the same time or nearly so. Using the offset of the light sources in the image horizontal plane, the horizontal distance between the two sensors of the stereo camera 102, and the focal length of the stereo camera 102, the first processing unit 101 calculates the distance from the stereo camera to the light source(s) and thus to the object, for example, to the vehicle ahead or a sign.

[0045] According to a further embodiment of the invention, in Fig. 2. A street in the vicinity of vehicle 200 is shown from the perspective of the driver of vehicle 200. Therefore, only the front of vehicle 200 is shown. Fig. 2 is shown. Another vehicle 204 is shown ahead. Fig. Figure 2 shows the vehicle 200, which has a computing unit 210 and a stereo camera 211. These two components of a device are connected inside the vehicle via line 212. From the perspective of the driver of vehicle 200, the LED sign 213 is shown, which has two light sources 206 and 207. The sign 213 is firmly fixed to the ground next to the road by means of mounting posts. The road user 201, i.e., a vehicle 201 traveling ahead, has two red taillights 208 and 209. With a simple, state-of-the-art light assistance system, however, both objects—the sign 213 and the road user 201—would be detected as vehicles traveling ahead.In the event that the two lights 206 and 207 of the LED sign 213 have a high red component in their color, and since they look the same and are at the same height, they would be paired by a state-of-the-art system and thus identified as a false positive vehicle detection.

[0046] However, the method and device according to the present invention prevent such false positive detection of objects like the LED sign 213. Due to the verification according to the invention as to whether the detected object is a road user, wherein the verification is carried out by means of a distance calculation, such an error can be avoided. In particular, a second distance calculation based on a three-dimensional position calculation of the detected object can be used. This makes it possible to determine that the detected object is not a vehicle ahead. Specifically, a comparison of two distance values ​​can be carried out, for example, using a first estimated distance value and using a distance value from a three-dimensional position calculation.This makes it possible to determine whether the paired lights and the object assigned to those paired lights could actually be a vehicle.

[0047] According to a further embodiment of the invention, a method for the verified detection of a road user is based on the detection of at least one light source in an image of a vehicle's surroundings. Fig. 3 shown. The procedure of Fig. Step 3 involves providing at least one image of the vehicle's surroundings, which occurs in step S1. In step S2, an object in the vehicle's vicinity is detected based on the detection of a light source from the object in the first image. Verification of whether the detected object is a road user, performed by calculating a distance, is carried out in step S3.

[0048] According to a further embodiment of the method Fig. In a further step S4, a pairing of light sources contained in the first image is performed, thereby detecting the object containing the light sources. Likewise, these two previously mentioned embodiments can be supplemented and extended by further process steps described in the context of the present invention. In particular, a driver assistance system, a light assistant, or a device for controlling the front lighting of a vehicle can perform these processes.

[0049] It should also be noted that "comprehensive" does not exclude any other elements or steps, and "a" or "an" does not exclude a plurality. Furthermore, it should be noted that features or steps described with reference to one of the above embodiments may also be used in combination with other features or steps from other embodiments described above. Reference numerals in the claims are not to be considered as limitations.

Claims

[1] Method for the verified detection of a road user based on the detection of at least one light source in an image of a vehicle's surroundings, the method further comprising the steps Providing at least one initial image of the vehicle's surroundings (S1), Detecting an object in the vicinity of the vehicle based on the detection of a light source of the object in the first image (S2), Verify whether the detected object is a road user, whereby verification is carried out by means of a distance calculation (S3), Determining an initial distance value between the vehicle and the detected object based on the first image, Determining a second distance value of the vehicle to the detected object based on a second image of the vehicle's surroundings, and where the verification of the object detection is carried out by comparing the first and second distance values. [2] Method according to claim 1, further comprising the step of performing a pairing of light sources contained in the first image, whereby the object containing the light sources is detected (S4). [3] Method according to one of claims 1 or 2, where the distance calculation is performed as a stereo distance calculation based on a second image of the vehicle's surroundings, and where the first image and the second image are generated by a stereo camera of the vehicle. [4] Method according to one of the preceding claims, the method further comprising the step of determining the second distance value based on a different horizontal position of the light sources in the first and the second image. [5] Method according to one of the preceding claims, wherein a three-dimensional position calculation of the detected object is carried out by means of disparities of the light sources with respect to the first and second image. [6] Method according to one of the preceding claims, wherein the determination of the first distance value is carried out as an estimation of the first distance value by means of a distance between two light sources of the object in the first image and by means of an average width of motor vehicles. [7] Method according to any of the preceding claims, the method further comprising the step of controlling the vehicle light based on a result of verifying whether the detected object is a road user. [8] Device (100) for a vehicle (200) for the verified detection of a road user (201) based on the detection of at least one light source, comprising the device a computing unit (101), wherein the computing unit is designed to verify the detection of an object in the vehicle's environment based on the detection of a light source of the object in a first image, and wherein the computing unit is designed to perform verification by calculating the distance between the detected object and the vehicle, wherein the computing unit is designed to determine an initial distance value of the vehicle to the detected object based on the first image, wherein the computing unit is designed to determine a second distance value of the vehicle to the detected object based on a second image of the vehicle's surroundings, and the computing unit is designed to perform the verification of the object detection by comparing the first and second distance values. [9] Device according to claim 8, wherein the device is designed to control a front light of the vehicle (200), the device further featuring a stereo camera (102), wherein the stereo camera includes a first sensor (103) for generating the first image (105) of the vehicle's surroundings (200), wherein the stereo camera includes a second sensor (104) for generating the second image (106) of the vehicle's surroundings (200), and the computing unit is designed to perform a three-dimensional position calculation of the detected object using disparities of the light sources with respect to the first and second images.

Images