Method for the optical identification of objects for a motor vehicle

Abstract

Method for the optical identification of an object (15) for a motor vehicle (1), wherein a time-limited first light pulse (LP1) is emitted in the direction of an object (15) by at least one first headlight (FS1) of the motor vehicle (1), wherein during the first light pulse (LP1) a first image of a spatial area illuminated by the first light pulse (1) is produced by means of at least one first camera (12) of the motor vehicle (1), wherein the first light pulse (1) is synchronized with an exposure time window of the first camera (12) such that at least 50% of a quantity of light of the first light pulse (LP1) is emitted during the exposure time window, and wherein an identification of a nature and / or traffic information of the object (15) is carried out on the basis of image recognition applied to the first image, wherein preliminary information (20) regarding the object (15) is obtained.which constitutes a trigger event with respect to the detection of the object (15), wherein, as a result of the trigger event, the first light pulse (LP1) is emitted and, synchronized with this, the first image of the spatial area illuminated by the first light pulse (LP1) is generated, characterized in that, as preliminary information (20) - from a digital map (18) the presence of a road feature and / or a traffic sign and / or - by means of a radar sensor (6) and / or a LiDAR sensor (8) the presence of an indeterminate object in a specific spatial area and / or - by means of a data connection (22) position information of an indeterminate object and / or - by means of the said first camera (12) and / or a further camera a preliminary image of an indeterminate object is obtained, and that in the case,that position information of an indeterminate object is obtained by means of a data connection (22) and / or a preliminary image of an indeterminate object is obtained by means of the aforementioned first camera (12) and / or a further camera, - based on the preliminary information (20) an at least approximate position of the object (15) is determined, and - by means of the first light pulse (LP1) a spatial area around the aforementioned position of the object (15) is selectively illuminated.

Description

[0001] The invention relates to a method for the light-optical identification of an object for a motor vehicle, wherein a light signal is emitted in the direction of an object by a headlight of the motor vehicle, a first image of a spatial area illuminated thereby is generated by means of a camera of the motor vehicle, and an identification of the object takes place on the basis of image recognition applied to the first image.

[0002] Cameras, LiDARs (light detection and ranging or laser imaging, detection, and ranging), and radars are currently predominantly used as sensors in advanced driver assistance systems for motor vehicles (EVs) and automated driving systems for vehicles. Both radar-based sensors and LiDARs only measure distances but cannot distinguish colors. Although these sensors can detect all types of objects, the inability to distinguish colors significantly reduces the ability to precisely identify, and in particular, to classify or categorize objects.

[0003] Light-optical sensors, i.e., sensors in the visible light spectrum such as cameras, are better suited for such precise object identification, preferably in combination with artificial intelligence algorithms for image recognition. However, systems with such sensors suffer from severe limitations in adverse visibility and / or weather conditions such as darkness, rain, fog, sun, or snow, as well as in sudden changes in conditions that can impair the lighting conditions and consequently also the visibility (as well as the contrast between colors) for cameras. Currently, a vehicle's headlights are often insufficient to provide sufficient contrast for purely camera-based object recognition at night, which can reduce accuracy by 40–60%.The resulting lack of robustness in a dynamic environment, i.e. reliability in the face of changing visibility conditions, remains a major disadvantage for vision-based driver assistance systems.

[0004] This poses a problem, not least for the development of automatically connected driving systems that could reach SAE Level 3 or higher, because current regulations require that multiple sensors must confirm information such as identified objects in the vicinity of a vehicle, but objects that could only be identified by one sensor (e.g. LiDAR in low-light conditions) cannot be officially confirmed.

[0005] The invention is therefore based on the object of specifying a method for the light-optical identification of an object for a motor vehicle, which should allow the best possible image recognition even in adverse lighting and / or visibility conditions.

[0006] The stated object is achieved according to the invention by a method for the light-optical identification of an object for a motor vehicle, wherein a time-limited first light pulse is emitted in the direction of an object by at least one first headlight of the motor vehicle, wherein during the first light pulse a first image of a spatial area illuminated by the first light pulse is generated by at least one first camera of the motor vehicle, wherein the first light pulse is synchronized with an exposure time window of the first camera such that at least 50% of a light quantity of the first light pulse is emitted during the exposure time window, and wherein an identification of a nature and / or traffic information of the object takes place based on image recognition applied to the first image. Advantageous and partly inventive embodiments are the subject of the dependent claims and the following description.

[0007] A headlight is understood in particular to be a headlight that is designed and configured to illuminate the road ahead of the vehicle during normal driving in the dark. In particular, the vehicle may have two headlights (left and right front), in which case the first light pulse is emitted by at least one of the two headlights (or by both simultaneously). However, the vehicle may also have a continuous "light bar" across the entire front as the first headlight.

[0008] A light pulse is understood here, in particular, to be a short light signal, preferably with a duration of at least 100 µs and / or at most 20 ms, which preferably has a uniform or largely uniform radiant power over the pulse duration. If the first headlight has a plurality of light elements, such as a matrix of LEDs, the radiant power across the individual light elements is preferably homogeneous or largely homogeneous over the pulse duration, and / or the radiant power of individual light elements is uniform or largely uniform over the pulse duration (where the term "substantial" encompasses a relative deviation of less than 10%).

[0009] In particular, the associated headlight can be used before and / or after the respective light pulse, even in normal operation (low beam or high beam), to illuminate the roadway, so that the light pulse consists of a brief increase in radiant power, and after the light pulse, the aforementioned normal operation can be continued. Preferably, the radiant power for the light pulse is generally at least 50% greater, especially in the present case of a brief increase, than the average radiant power of the first headlight in high beam mode.

[0010] The first light pulse can be emitted frontally by the first headlight without any specific directional effect, so that an object located somewhere in front of the vehicle can be illuminated by the first light pulse and thus captured via the first image recording.

[0011] However, the first light pulse can also be directed to illuminate the spatial area of ​​an object based on preliminary information that already documents its presence. In this case, the previously unidentified object can be precisely identified using the method, both in terms of its nature and / or traffic information.

[0012] The exposure time window of the first camera, i.e., the exposure time interval relevant for generating the first (and possibly further) images captured by the first camera, is synchronized with the first light pulse of the first headlight (and possibly with further light pulses) such that the first light pulse falls at least predominantly, i.e., at least 50% of its emitted light quantity, and preferably completely within said exposure time window. The length of the first light pulse is thus preferably comparable to that of the exposure time window, and in particular, should be selected to be identical or shorter.The synchronization is preferably carried out by a corresponding synchronization line between the first headlight and the first camera, by a synchronization line between corresponding control units of the first headlight and the first camera, or via a control unit to which both the first headlight and the first camera (or control units of the first headlight and the first camera) are connected.

[0013] Automatic image recognition, which is based in particular on artificial intelligence and / or a machine learning algorithm, can now be applied to the first image recording in order to identify the nature and / or traffic information of the object and, if necessary, to categorize the object accordingly. The nature of the object can in particular include a distinction between a traffic sign or other sign (such as traffic lights), a person or animal, another motor vehicle or other vehicle (if necessary with a distinction between a truck, motorcycle, bicycle, etc.), a branch or stone, etc. The traffic information of the object can in particular include the content of a traffic sign or a variable message sign or another display addressed to motor vehicle drivers. The identified nature and / or traffic information of the object is then preferably used to plan the driving movement (so-calledMotion Planning), for example for reducing the driving speed of the vehicle or similar.

[0014] Preferably, preliminary information regarding the object is obtained, which forms a triggering event regarding a detection of the object, wherein as a result of the triggering event the first light pulse is emitted and, synchronized therewith, the first image recording of the spatial area illuminated by the first light pulse is generated.

[0015] In particular, the provisional information includes the presence of a road feature and / or a traffic sign from a digital map and / or the presence of an as yet undetermined object in a specific spatial area, which was detected by means of a radar sensor and / or a LiDAR sensor, and / or position information of an undetermined object by means of a data connection and / or a provisional image recording of an undetermined object by means of the said camera and / or another camera.

[0016] The digital map from which the road feature is obtained can be stored locally in the vehicle (e.g., in an offline navigation system) or retrieved from a corresponding server via a wireless data connection. The radar and / or LiDAR sensor can abstractly detect the presence of an object in a specific spatial area, with this detected presence triggering a check of the object in order to specifically identify it in the aforementioned sense. A similar situation applies to position information, which can be transmitted from another vehicle, e.g., via LoRaWAN or similar. The received position and, if applicable, the type of obstacle (here: another vehicle) can then trigger confirmation of the identification described above.

[0017] In this case, an at least approximate position of the object is expediently determined on the basis of the preliminary information, and a spatial area around the said position of the object is illuminated by the first light pulse. In particular, the preliminary information can also already comprise at least an approximate position indication of the object to be identified, as in the cases described above. The position indication, e.g. from a digital map, a radar / LiDAR image, or a preliminary camera image recording, can then be used to specifically align the first light pulse to the position of the object and to specifically identify the object and a (preferably limited orsmall) area of ​​space directly surrounding the object, i.e. to concentrate the light output as precisely as possible on the object in order to facilitate identification and also to avoid any impairment of other road users by the first light pulse.

[0018] For this purpose, the first headlight preferably has a plurality of light elements with an adjustable radiation direction, wherein the spatial area around the said position of the object is illuminated by a corresponding alignment of the light elements of the first headlight in the direction of the said position of the object. Light elements with an adjustable radiation direction particularly includes the fact that the respective direction of maximum radiation intensity can be varied separately from one another across the cross-sectional area of ​​the first headlight from at least two different regions (which are each assigned to different light elements). This can be achieved, for example, by an arrangement of a plurality of adjustable mirrors and / or a plurality of adjustable LEDs (such as a matrix of individually movable LEDs).

[0019] After the end of the first light pulse, a second headlight of the motor vehicle expediently emits a time-limited second light pulse in the direction of the object, wherein during the second light pulse a second image of the spatial area illuminated by the second light pulse is generated by the first camera of the motor vehicle, wherein the second light pulse is synchronized with an exposure time window of the first camera such that at least 50% of a light quantity of the second light pulse is emitted during the exposure time window, and wherein the identification of the nature and / or traffic information of the object is also carried out using image recognition applied to the second image recording.

[0020] In particular, the first light pulse is emitted by the first headlight alone, and the second light pulse by the second headlight alone. Preferably, after the end of the second light pulse, a third light pulse is emitted by the first and second headlights, synchronized with each other and with the exposure window in the manner described above. During this third light pulse, a third image is captured by the camera, which is also used to identify the nature and / or traffic information of the object.

[0021] The aforementioned identification of the object can then be performed based on the first and second (and possibly the third) image recordings. In particular, the advantage can be exploited that the object experiences different illumination due to the first and second (and possibly the third) light pulses, so that spatial information (such as the object's shape) can also be obtained from the images.

[0022] The invention further relates to a motor vehicle having at least one first headlight, a camera, and a control unit, which is configured to carry out the method described above. The method is controlled in particular by the control unit, which preferably synchronizes the first and, if applicable, further light pulses with the exposure time window of the first and, if applicable, further image recordings.

[0023] The vehicle according to the invention shares the advantages of the method according to the invention. The advantages stated for the method and its further developments can be applied analogously to the vehicle.

[0024] An embodiment of the invention is explained in more detail below with reference to the accompanying drawings. The drawings schematically show: Fig. 1 in a block diagram a vehicle with a synchronization between a camera and the headlights.

[0025] In Fig.1 schematically shows a motor vehicle 1 in a block diagram. The motor vehicle 1 comprises a first headlight FS1, which is arranged at a left front corner 2 of the motor vehicle 1. The motor vehicle 1 further comprises a second headlight FS2, which is arranged at a right front corner 4 of the motor vehicle 1. The motor vehicle 1 also comprises a radar sensor 6 and a LiDAR sensor 8, both of which are controlled by a sensor control unit 10. The motor vehicle 1 also has a first camera 12, which is connected to a control unit 14, by means of which, in a manner to be described below, the detection of objects in front of the motor vehicle 1 during driving is controlled.In addition, the motor vehicle 1 has a navigation system 16 in which a number of digital maps 18 are stored, wherein the digital maps 18 each represent different geographical regions, and additional information regarding the roads running in the respective regions is also recorded in the digital maps 18 (for example traffic signs and symbols, traffic lights, speed limits, etc.).

[0026] During normal driving operation, the control unit 14 is designed, among other things, to detect objects in front of the motor vehicle 1 and identify them by their nature, thus in particular to enable categorization of the objects for subsequent control of the driving operation, and if necessary to obtain traffic information from an object (for example, if a traffic sign or a variable-speed signal system is used as an object). This applies in particular when the movement of the motor vehicle 1 is automated from an SAE level 3 or higher. For the aforementioned identification of an object, data from the radar sensor 6 and / or the LiDAR sensor 8 can be used, which is recorded in the sensor control unit 10 as preliminary information 20 regarding such an object. In this case, for example, the presence of an object 15 in front of the motor vehicle 1 and an approximate position of the detected object 15 can be detected using the radar sensor 6.Likewise, the presence and distance of the object 15 can be detected using the LiDAR sensor 8. However, the sensor signals from these sensors cannot resolve color information about the object 15. This means that, particularly in the case of traffic signs or variable-signal systems, reliable identification of the traffic information contained in the object cannot be guaranteed using these sensors alone. Furthermore, such color information is particularly important for precise identification of unknown objects such as branches or other natural obstacles that randomly appear on the road (such as wildlife).

[0027] However, the aforementioned preliminary information 20 regarding the presence of object 15 in front of motor vehicle 1 can also be obtained from a digital map 18. In this case, for example, the navigation system 16 detects that motor vehicle 1 is located in the area of ​​a traffic-relevant object. However, this preliminary information 20 regarding the presence of the object still requires confirmation for reliable use of traffic information associated with object 15. Likewise, the preliminary information 20 regarding the presence of a traffic-relevant object can be transmitted externally to a communication device 22 of motor vehicle 1, for example, from another vehicle using LoRaWAN or similar protocols.The communication device 22, which is preferably equipped with a corresponding antenna and other hardware required for communication, can then forward the preliminary information 20 to the control unit 14 with a corresponding request for confirmation. In particular, the navigation system 16 can also use the communication device 22 to determine the position of the motor vehicle 1 as part of navigation planning (dashed line).

[0028] Now that the preliminary information 20 regarding the object 15 in front of the motor vehicle 1 is available, the control unit 14 initiates a process by means of which the nature of the object 15 is identified and / or specific traffic information about the object 15 is identified. For this purpose, a first alignment command 23 is sent to the first headlight FS1 via a first control line 21. In response to this first alignment command 23, individual light elements 31 of the first headlight FS1 are aligned in the direction of the object 15 according to the position as determined by the preliminary information 20. Furthermore, the control unit 14 transmits a second alignment command 27 to the second headlight FS2 via a second control line 26, by means of which second alignment command 27, light elements (not shown in detail) of the second headlight FS2 are aligned to the detected orbe aligned towards the provisionally estimated position of object 15, so that a later light pulse radiates its maximum power towards object 15 and illuminates it accordingly in the best possible way.

[0029] Furthermore, after the first alignment command 23, the control unit 14 outputs a first control command 24 to the first headlight FS1 via the first control line 21, as a result of which the individual light elements 31 (as described by the first alignment command 23, aligned with the object 15 based on the preliminary information 20) emit a first light pulse LP1. The first light pulse LP1 has a duration in the millisecond range and is synchronized by the control unit 14 with a first image captured by the camera 12, with the control unit 14 outputting a first image command 28 to the camera 12 for this purpose.By synchronizing the first light pulse LP1 with the first image acquisition via the first control command 24 and the first image command 28, it is achieved that a large part, at least half and in particular all of the radiation energy of the first light pulse LP1 falls into an exposure time window of the first camera 12 for the first image acquisition.

[0030] In a similar manner, after the end of the first light pulse LP1, a second control command 29 is output to the second headlight FS2 via the second control line 26, as a result of which the light elements of the second headlight FS2 emit a second light pulse LP2. Furthermore, the control unit 14 outputs a second image command 30 to the first camera 12, which is synchronized with the second control command 29 such that a second image captured by the first camera 12 as a result of the second image command 30 is synchronous with the second light pulse LP2 of the second headlight FS2 (and at least half, and preferably all, of the radiant energy of the second light pulse LP2 falls within an exposure time window for the second image capture or is emitted during it).

[0031] After the end of the second light pulse LP2, the control unit 14 outputs a third control command 32 to both the first headlight FS1 and the second headlight FS2 (i.e., via both the first control line 21 and the second control line 26). Furthermore, a third image command 33 is output to the first camera 12 synchronously (in the above sense) with the third control command 32. The first and second headlights FS1, FS2 each generate a third light pulse LP3 as a result of the third control command 32, while synchronously with the third light pulse LP3, the first camera 12 generates a third image as a result of the third image command 33, the exposure time window of which is selected such that at least half, and in particular all, of the radiant energy of the third light pulse LP3 is emitted by the two headlights FS1, FS2 during the third image acquisition.The control of the headlights FS1, FS2 for the light pulses LP1, LP2, LP3 can also be done indirectly by the control unit 14 first controlling a control unit (not shown) associated with the respective headlight, which then specifically controls the respective light pulse of the headlight.

[0032] The light output of the light pulses LP1, LP2, LP3 is stronger and in particular significantly stronger than an average light output of the headlights FS1, FS2 in normal operation, which also includes high beam operation.

[0033] Because the additional radiation output lasts only for a short period of a few milliseconds, any impairment and, in particular, endangerment of other road users (such as drivers of oncoming vehicles) can be avoided. Due to the increased light output and, in particular, the spatial alignment of the light elements of the headlights FS1, FS2 toward object 15 and the corresponding targeted illumination of object 15, high contrast and high resolution in the aforementioned images can be achieved, especially in situations with poor visibility in the surrounding area.The increase in radiant power during the three light pulses LP1, LP2, LP3 can be switched on selectively in normal operation (i.e. the low beam or the high beam) so that after the end of the said light pulses LP1, LP2, LP3, the headlights FS1, FS2 each switch back to the normal operation mode that existed before the image recording.

[0034] Based on the image capture, an image recognition algorithm, which is based in particular on machine learning and / or artificial intelligence, can now identify the nature of the object 15 and / or traffic information received from the object 15, and this information can be used for color planning, particularly in automated driving. The identification can preferably be performed in the control unit 14 or in a dedicated computing unit (not shown in detail).

[0035] Although the invention has been illustrated and described in detail by the preferred embodiment, the invention is not limited to the disclosed examples and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. LIST OF REFERENCE SYMBOLS: 1 vehicle 2 left front corner 4 right front corner 6 radar sensor 8 LiDAR sensor 10 Sensor control unit 12 first camera 14 Control unit 15 objects 16 Navigation system 18 digital map 20 preliminary information 21 first control line 22 Communication device 23 first alignment command 24 first control command 26 second control line 27 second alignment command 28 first image command 29 second control command 30 second image command 31 light elements (of the first headlight) 32 third control command 33 third image command FS1 / 2 first / second headlight LP1 / 2 / 3 first / second / third light pulse

Claims

[1] Method for the light-optical identification of an object (15) for a motor vehicle (1), wherein a time-limited first light pulse (LP1) is emitted in the direction of an object (15) by at least one first headlight (FS1) of the motor vehicle (1), wherein during the first light pulse (LP1) a first image recording of a spatial area illuminated by the first light pulse (1) is generated by means of at least one first camera (12) of the motor vehicle (1), wherein the first light pulse (1) is synchronized with an exposure time window of the first camera (12) such that at least 50% of a light quantity of the first light pulse (LP1) is emitted during the exposure time window, and wherein an identification of a nature and / or traffic information of the object (15) is carried out on the basis of image recognition applied to the first image recording. [2] Method according to claim 1, wherein preliminary information (20) regarding the object (15) is obtained, which constitutes a trigger event regarding a detection of the object (15), and wherein, as a result of the triggering event, the first light pulse (LP1) is emitted and, synchronized therewith, the first image recording of the spatial area illuminated by the first light pulse (LP1) is generated. [3] Method according to claim 2, where as preliminary information (20) - from a digital map (18) a presence of a road feature and / or a traffic sign and / or - by means of a radar sensor (6) and / or a LiDAR sensor (8) the presence of an indeterminate object in a specific spatial area and / or - by means of a data connection (22) a position information of an indeterminate object and / or - a preliminary image of an indeterminate object is obtained by means of said first camera (12) and / or a further camera. [4] Method according to claim 3, wherein an at least approximate position of the object (15) is determined on the basis of the preliminary information (20), and wherein the first light pulse (LP1) specifically illuminates a spatial area around the said position of the object (15). [5] Method according to claim 4, wherein the first headlight (FS1) has a plurality of light elements (31) with adjustable beam direction, and wherein the spatial area around said position of the object (15) is illuminated by a corresponding alignment of the light elements (31) of the first headlight (LP1) in the direction of said position of the object (15). [6] Method according to one of the preceding claims, wherein after the end of the first light pulse (LP1) a time-limited second light pulse (LP2) is emitted in the direction of the object (15) by a second headlight (FS2) of the motor vehicle (1), wherein during the second light pulse (LP2) a second image recording of the spatial area illuminated by the second light pulse (LP2) is generated by means of the first camera (12) of the motor vehicle, wherein the second light pulse (LP2) is synchronized with an exposure time window of the first camera (12) such that at least 50% of a light quantity of the second light pulse is emitted during the exposure time window, and wherein the identification of the nature and / or traffic information of the object (15) is also carried out using image recognition applied to the second image recording. [7] Method according to one of the preceding claims, wherein the first light pulse (LP1) has a duration of at least 100µs and / or of at most 20 ms. [8] Method according to one of the preceding claims, wherein a radiation power of the first light pulse (LP1) is at least 50% greater than an average radiation power of the first headlight in a high beam mode. [9] Method according to one of the preceding claims, wherein the identified nature and / or traffic information of the object (15) is used for planning the driving movement. [10] Motor vehicle (1) with at least one first headlight (FS1), a first camera (12) and a control unit (14), which is designed to carry out the method according to one of the preceding claims.

Images