Method for measuring the distance between an object and an optical sensor, control device for carrying out such method, distance measuring device having such a control device, and motor vehicle having such a distance measuring device

Abstract

The invention relates to a method for measuring the distance between an object (17) and an optical sensor (7) by means of an illumination device (5) and an optical sensor (7), wherein the actuation of the illumination device (5) and the optical sensor (7) is coordinated with one another in time, wherein the local position of a visible distance region (15) within the observation region (13) of the optical sensor (7) is obtained by the temporal coordination of the actuation of the illumination device (5) and the optical sensor (7), wherein a photograph (23) of the visible distance region (15) is obtained by means of the coordinated actuation of the illumination device (5) and the optical sensor (7). The invention relates to a method for controlling a distance range (15) and photographing the object (17) by controlling the image line, wherein a starting image line for the starting point (19) of the visible distance range (15) and an end image line for the end point (21) of the visible distance range are determined in the photograph (23), wherein in the photograph (23), a reference point image line is determined as the image line with the shortest distance to the starting image line, in which the object (17) can be detected, and wherein the distance to the object (17) is determined taking into account the local position of the visible distance range (15) by evaluating the image position of the reference point image line relative to the starting image line and the end image line.

Description

Technical Field

[0001] The invention relates to a method for measuring the distance between an object and an optical sensor, a control device designed to carry out such a method, a distance measuring device having such a control device, and a motor vehicle having such a distance measuring device. Background Art

[0002] A method for measuring the distance between an object and an optical sensor using an illumination device and an optical sensor is known. This method is disclosed in International Patent Application Publication No. WO 2017 / 009848 A1, in which the illumination device and the optical sensor are controlled in a temporally coordinated manner to capture a specific visible distance range within the field of view of the optical sensor, wherein the visible distance range results from the temporal coordination of the control of the illumination device and the optical sensor.

[0003] The problem here is to determine the reference point distances of the identified objects within the distance range, in particular when the vehicle carrying out the method experiences its own pitching motion and / or the road on which the vehicle is located has a road course that deviates from the horizontal or has at least a varying angle of attack relative to the horizontal. Summary of the Invention

[0004] The object of the present invention is therefore to provide a method for measuring the distance between an object and an optical sensor, a control device designed to carry out such a method, a distance measuring device having such a control device, and a motor vehicle having such a distance measuring device, wherein the aforementioned disadvantages are at least partially eliminated, preferably avoided.

[0005] This object is achieved in that the technical teaching herein is provided, in particular the teaching of the independent claim and the embodiments disclosed in the dependent claims and the description.

[0006] In particular, this object is accomplished by providing a method for measuring the distance between an object and an optical sensor using an illumination device and an optical sensor. The illumination device and the optical sensor are controlled in a temporally coordinated manner. The local position of a visible distance range within the observation area of the optical sensor is determined by temporally coordinating the control of the illumination device and the optical sensor. Using the coordinated control, a photograph of the visible distance range is taken using the optical sensor. A starting image line is determined in the photograph for the starting point of the visible distance range. Furthermore, an end image line is determined in the photograph for the end point of the visible distance range. A reference point image line is determined in the photograph, wherein the reference point image line is an image line in which, on the one hand, an object can be detected and, on the other hand, has the shortest distance to the starting point image line within the image plane of the optical sensor. Finally, the distance to the object is determined by evaluating the image position of the reference point image line relative to the starting and end image lines, taking into account the local position of the visible distance range.

[0007] The method presented herein advantageously makes it possible to determine the distance of an object from the optical sensor, in particular the so-called reference point distance or so-called reference point distance, with high precision and at least substantially independent of the pitching motion of the vehicle executing the method and / or the angle of attack of the roadway on which the object is located relative to the horizontal. This is particularly possible because the reference point distance of the object is determined by interpolation from the image positions of the reference point image lines relative to the starting and end image lines within the visible distance range. In other words, the local position of the visible distance range in real space (i.e., on the object side) is known from the temporal control of the lighting device, on the one hand, and the optical sensor, on the other hand, while the image area of the distance range on the optical sensor is known by determining the starting and end image lines, on the one hand. It is therefore advantageous to find the reference point image lines and to infer the spatial object-side position of the object within the visible distance range on the object side from their positions within the image area relative to the starting and end image lines, i.e., the image positions. When the angle of attack of the road course relative to the horizontal plane changes, and also when the angle of the optical sensor relative to the road changes, for example due to its own pitch movement, the image area of the visible distance range on the optical sensor changes particularly significantly, which is easily and inherently taken into account by the method proposed here, and therefore the correct result of the evaluation proposed here is not affected.

[0008] This method can be particularly advantageously used in autonomous vehicles, particularly autonomous trucks. In particular, when driving behind a preceding vehicle without high travel distance requirements, this method can be used to advantageously detect objects in the vehicle's lane that cannot be driven over, particularly objects that are small compared to the vehicle's size. The method allows for timely and appropriate responses to the detection of such objects, particularly the determination of the vehicle's distance from the object. Such appropriate responses can include, for example, sudden braking or driving through a particularly specific avoidance trajectory, if necessary.

[0009] Such small objects that cannot be run over are generally also referred to as "lost goods". However, such objects can also be people or animals lying on the road, especially those that have been in an accident.

[0010] Methods for producing images by means of temporally coordinated activation of an illumination device and an optical sensor are particularly known as gated imaging methods. In particular, the optical sensor is a camera that is sensitively switched only within a specific, limited time range, which is referred to as gated activation; the camera is therefore a gated camera. Accordingly, the illumination device is also activated temporally only within specific, selected time periods to illuminate the scene on the object side.

[0011] In particular, a predetermined number of light pulses, preferably with a duration between 5 ns and 20 ns, is emitted by the illumination device. The start and end of the exposure of the optical sensor are correlated with the number and duration of the emitted light pulses. Thus, a specific visible distance range can be determined by the optical sensor via the temporal control of the illumination device on the one hand and the optical sensor on the other hand, at a correspondingly defined local position, i.e., in particular, a specific distance from the starting point of the distance range to the optical sensor and a specific distance range width.

[0012] Here, the visible distance range is the area on the object side in three-dimensional space, which is represented in a two-dimensional photograph on the image plane of the optical sensor by means of the number and duration of the light pulses of the lighting device in combination with the start and end point of the exposure of the optical sensor.

[0013] In contrast, the observation region is, in particular, the region on the object side in three-dimensional space, which, when adequately illuminated and exposed by the optical sensor, can be generally and particularly best represented in a two-dimensional photograph by the optical sensor. In particular, the observation region corresponds to the entire, theoretically illuminated, exposable image area of the optical sensor. The visible distance range is therefore a subset of the observation region in real space. Accordingly, in the method presented here, only a subset of the image plane of the optical sensor is exposed, wherein, in particular, a partial region of this image plane exists between the starting and ending image lines.

[0014] When we refer to the "object side" here and below, we are referring to the area in real space, that is, the side of the object to be observed. When we refer to the "image side" here and below, we are referring to the area in the image plane of the optical sensor. The observation area and the visible distance area are located on the object side. Corresponding to this is the image-side area in the image plane that is assigned due to the imaging theorem and the temporal control of the lighting device and the optical sensor.

[0015] The photons of the light pulse strike the optical sensor at the start and end of its exposure after the illumination by the illumination device begins. The further the visible distance range is from the illumination device and the optical sensor, the longer the time it takes for reflected photons in that distance range to strike the optical sensor. Therefore, the further the visible distance range is from the illumination device and the optical sensor, the longer the time difference between the end of illumination and the start of exposure.

[0016] According to one embodiment of the method, it is therefore possible in particular to define the position and spatial extent of the visible distance range by correspondingly suitable selection of the temporal activation of the lighting device on the one hand and the optical sensor on the other hand.

[0017] In an alternative embodiment of the method, a visible distance range can be specified, wherein the temporal coordination of the lighting device, on the one hand, and the optical sensor, on the other hand, is determined therefrom and set accordingly.

[0018] In this context, an image line is understood to mean, in particular, the set of all pixels of a picture in an image plane of the optical sensor, which pixels are located on the same horizontal line in the image plane.

[0019] In a preferred embodiment, the lighting device is a laser. In a preferred embodiment, the optical sensor is a camera.

[0020] The reference point image line is preferably determined as follows: Object recognition is performed in the photograph, in particular by pattern recognition, preferably using a classification algorithm, and / or with the aid of deep learning. If an object is recognized, all image lines in the photograph that depict the object are determined based on this recognition or classification. The image line with the shortest distance to the starting point image line is then determined as the reference point image line.

[0021] This method advantageously allows, in particular, the determination of the distance between an object and the optical sensor from a single photograph.

[0022] The distance between the object and the optical sensor is therefore preferably determined from a single image. However, a plurality of images can be included in the evaluation in order, in particular, to reduce measurement errors and / or to increase the accuracy of the distance determination.

[0023] According to a refinement of the present invention, for a photograph of a distance range, a row histogram is created for all image lines assigned to an evaluation region within the observation range on the optical sensor using the sum of the illumination intensities of each image line of the optical sensor. The starting and ending image lines are then determined using the row histogram. This advantageously allows the image positions of the region on the optical sensor that is assigned to the visible distance range on the object side to be determined on the image side. In other words, the temporal control of the lighting device, on the one hand, and the optical sensor, on the other hand, results in sharp brightness transitions at the starting and ending points of the image-side distance range. This ultimately allows the object distance to be determined by interpolating the positions of the reference point image lines relative to the starting and ending image lines.

[0024] In this context, a line histogram means, in particular, that each image line of the optical sensor within the evaluation area is assigned the sum of the illumination intensities of all pixels within the evaluation area for the respective image line. In this way, brightness transitions that occur in response to the temporal control can be detected easily and reliably within the image plane of the optical sensor.

[0025] According to one preferred embodiment, the evaluation region is identical to the observation region. This corresponds to a particularly simple implementation of the method. However, it is also possible, according to another preferred embodiment, for the evaluation region to be selected to be smaller than the observation region, in particular, smaller than the target region (region of interest) where the object to be detected may be located. This advantageously allows for a faster and more efficient execution of the method.

[0026] By only including pixels located within the evaluation region in the summation, the evaluation region can also be limited horizontally, in particular.

[0027] Preferably, the evaluation area in the photograph is predicted by GPS before the calculation of the line histogram, in particular by back-projecting the road course onto the image plane and / or identified by an optical trajectory tracking method.

[0028] According to a refinement of the invention, the object distance is determined as the distance between the object and the optical sensor, wherein the distance zone width is determined as the difference between the end point of the visible distance zone and the starting point of the visible distance zone. The reference point distance is determined as the image line distance between the reference point image line and the starting point image line on the optical sensor. In addition, the distance zone image width is determined as the image line distance between the end point image line and the starting point image line. The object distance is then finally determined by multiplying the distance zone width by the ratio of the reference point distance to the distance zone image width and then adding it to the starting point of the visible distance zone, i.e., in particular, the starting point of the spatial distance between the starting point of the visible distance zone and the optical sensor. The object distance is determined in particular according to the following formula:

[0029]

[0030] Among them, x near is the starting point of the visible distance area, x far is the end point of the visible distance region, and accordingly, (x far -x near ) is the distance zone width, v near is the starting image row, v far is the end image row, and accordingly, (v far -v near ) is the width of the distance region image, v is the reference point image row, and accordingly, (vv near ) is the reference point distance, and x is the object distance.

[0031] This approach is ultimately based on the intersection theorem, in which two photographs are required for a single application. On the one hand, the road surface is assumed to be linear within the visible distance range. On the other hand, strictly speaking, the intersection theorem assumes that the imaginary lines connecting the starting image line and the starting point of the visible distance range, on the one hand, and the end image line and the end point of the visible distance range, on the other hand, are parallel to each other, which is generally not the case. However, the distance between the optical sensor and the visible distance range is usually large enough to assume that the corresponding imaginary lines are parallel in any case with a very close approximation, so that the intersection theorem can be applied with a very close approximation in any case. It follows that the ratio of "object distance minus the starting point of the visible distance range" to the width of the distance range is equal to the ratio of the reference point distance to the width of the distance range image. This relationship is then solved for the object distance, resulting in the above-mentioned formula (1).

[0032] According to a refinement of the present invention, the lighting device and the optical sensor are each designed to operate in the near-infrared range. This has the advantage that the eyes of people and / or animals accidentally struck by the light from the lighting device are not adversely affected. Wavelengths greater than 1.4 μm, particularly 1.55 μm, are particularly advantageous because they are strongly absorbed by the lens and cornea of the eye, resulting in only minimal intensity reaching the retina. This also has the advantage that other road users are not dazzled by the lighting device, particularly when driving at night.

[0033] According to a refinement of the present invention, a plurality of chronologically ordered photographs is created, wherein the temporal coordination of the lighting device and the optical sensor is modified such that the change in the distance of the object over time is determined. In particular, the temporal coordination of at least two photographs in the chronologically ordered photographs is modified. Particularly preferably, the temporal coordination of each photograph in the chronologically ordered photographs is modified. In particular, the temporal coordination of each photograph in the chronologically ordered photographs is modified such that the reference point image line remains approximately centered between the starting image line and the end image line. The change in the temporal coordination of the lighting device and the optical sensor, which is required for this purpose, can then be used to infer the change in the distance of the object over time. This advantageously allows for dynamic object distance measurement.

[0034] This object is also achieved by providing a control device designed to carry out the method according to the present invention or according to one of the above-described embodiments. The control device is preferably designed as a computing device, particularly preferably as a computer or a controller, in particular as a controller of a vehicle. The advantages already explained in conjunction with the method arise in particular with the control device.

[0035] This object is also achieved in the following manner, namely providing a distance measuring device having an illumination device, an optical sensor and a control device according to the invention or a control device according to one of the above-described embodiments. In particular, the advantages already explained in conjunction with the method and the control device result in the distance measuring device.

[0036] The control device is preferably operatively connected to the lighting device on the one hand and to the optical sensor on the other hand and is designed to control them.

[0037] Ultimately, this object is achieved by providing a motor vehicle having a distance measuring device according to the invention or a distance measuring device according to one of the aforementioned embodiments. In relation to the motor vehicle, in particular, the advantages already explained in connection with the method, the control device, and the distance measuring device result.

[0038] In an advantageous embodiment, the motor vehicle is designed as a truck. However, it is also possible that the motor vehicle is a passenger car, a commercial vehicle or another motor vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention is explained in detail below with reference to the accompanying drawings, in which:

[0040] Figure 1 A schematic diagram showing an exemplary embodiment of a motor vehicle having an exemplary embodiment of a distance measuring device,

[0041] Figure 2 shows a schematic diagram of a photograph taken with an optical sensor within the scope of one embodiment of the method,

[0042] Figure 3 A schematic diagram showing a row histogram used in an embodiment of the method. DETAILED DESCRIPTION

[0043] Figure 1 A schematic diagram of an exemplary embodiment of a motor vehicle 1 is shown, which has an exemplary embodiment of a distance measuring device 3. The distance measuring device 3 has an illumination device 5 and an optical sensor 7. The distance measuring device 3 also has a control device 9, which is shown only schematically here and is operatively connected to the illumination device 5 and the optical sensor 7 in a manner not explicitly shown, for corresponding control thereof. Figure 1 In particular, the illumination light cone 11 of the illumination device 5 and the observation range 13 of the optical sensor 7 are shown. Furthermore, a visible distance range 15 is shown hatched, which is present as a subset of the observation range 13 of the optical sensor 7 .

[0044] An object 17 is arranged within the visible distance range 15 .

[0045] exist Figure 1 The start point 19 and the end point 21 of the visible distance region 15 are also drawn in.

[0046] The control device 9 is designed, in particular, to carry out an embodiment of a method for measuring a distance x between an object 17 and the optical sensor 7 , as described in greater detail below.

[0047] Here, the lighting device 5 and the optical sensor 7 are controlled in a temporally coordinated manner, wherein the temporal coordination of the control of the lighting device 5 and the optical sensor 7 results in the local position of the visible distance range 15 within the observation area 13. A photograph of the visible distance range 15 is taken by the optical sensor 7 using the coordinated control.

[0048] Figure 2 A schematic diagram of such a photograph 23 is shown in the image plane of the optical sensor 7. Figure 2 The starting image line v for the starting point 19 of the distance range 15 visible in the photograph 23 is shown. near and the endpoint image row v for endpoint 21 far . Starting image row v near The sum of the end image rows v far The position of is determined. In addition, in photo 23, as a distance from the starting image row v near The image line with the shortest distance from the reference point determines the reference point image line v in which the object 17 can be detected. The distance of the object 17 is then determined by evaluating the reference point image line v relative to the starting point image line v. near and the endpoint image row v farThe image position of , ie its position in the photograph 23 , is determined taking into account the local position of the visible distance region 15 on the object side.

[0049] exist Figure 2 The image of the object 17 in the photograph 23 is denoted by 17 ′.

[0050] In addition, Figure 2 An evaluation region 27 is drawn in, which can be determined in particular by GPS prediction and / or optical trajectory tracking methods. The evaluation region 27 is here a target region that is smaller than the observation region 13. However, it can also coincide with it.

[0051] In particular, the object distance x (see Figure 1 ) is determined as the distance between the object 17 and the optical sensor 7 in the following manner, ie, the distance region width (x far -x near ) is determined as the difference between the end point 21 of the visible distance region 15 and the start point 19 of the visible distance region 15. near ) is determined as the position between the reference point image line v and the starting point image line v on the optical sensor 7 near The distance between the image rows. The distance between the image width (v far -v near ) as the end image row v far With the starting image row v near The distance between the image lines is found. The object distance x is then calculated by dividing the distance region width (x far -x near ) multiplied by the base point distance (vv near ) and the width of the distance region image (v far -v near ) is then added to the starting point 19 of the visible distance region 15. In particular, the object distance x is determined according to the above formula (1).

[0052] Figure 3 Show the basis Figure 2 Schematic diagram of a row histogram 25 of a photograph 23 or an evaluation region 27 of a photograph 23. In the row histogram 25, the individual image lines of the optical sensor 7 are plotted on the abscissa, wherein the sum of the illumination intensities of each pixel of all pixels in the evaluation region 27 for the corresponding image line is plotted on the ordinate. The row histogram 25 is created for all image lines assigned to the evaluation region 27 on the optical sensor 7 using the sum of the illumination intensities of each image line of the optical sensor 7. Starting image line v near and the endpoint image row v farIt is then determined with the aid of the line histogram 25, wherein, in particular because the lighting device 5 and the optical sensor 7 are controlled in a temporally coordinated manner, it can be detected that on the one hand the image line v at the starting point near and on the other hand at the end image row v far There is a noticeable jump in intensity.

[0053] The lighting device 5 and the optical sensor 7 are preferably designed to operate in the near-infrared range, in particular at 1.55 μm.

[0054] Within the scope of the method, a plurality of chronologically ordered photographs 23 are preferably created, wherein the temporal coordination of the lighting device 5 and the optical sensor 7 is varied so that a change in the distance of the object 17 over time can be determined.

Claims

1. A method for measuring the distance between an object (17) and the optical sensor (7) by means of an illumination device (5) and an optical sensor (7), wherein: The activation of the lighting device (5) and the optical sensor (7) are coordinated with each other in terms of time, By temporally coordinating the activation of the lighting device (5) and the optical sensor (7), a local position of a visible distance range (15) within the observation range (13) of the optical sensor (7) is obtained, wherein a photograph (23) of the visible distance range (15) is recorded by means of the coordinated activation with the optical sensor (7). In the photograph (23), a starting image line for the starting point (19) of the visible distance range (15) and an end image line for the end point (21) of the visible distance range are determined, In the photograph (23), the reference point image line is determined as the image line having the shortest distance from the starting point image line, in which the object (17) can be detected, and By evaluating the image position of the reference point image line relative to the starting image line and the end image line, the distance to the object (17) is determined taking into account the local position of the visible distance region (15).

2. The method according to claim 1, wherein For a photograph (23) of the distance range (15), a row histogram (25) of all image rows associated with an evaluation region (27) within the observation region (13) on the optical sensor (7) is created by means of the sum of the illumination intensities of each image row of the optical sensor (7), wherein the starting image row and the end image row are determined by means of the row histogram (25).

3. The method according to claim 1 or 2, wherein: The object distance is determined as the distance between the object (17) and the optical sensor (7), wherein the distance region width is determined as the difference between the end point (21) of the visible distance region (15) and the starting point (19) of the visible distance region (15), wherein the reference point distance is determined as the image line distance between the reference point image line and the starting point image line on the optical sensor (7), wherein the distance region image width is determined as the image line distance between the end point image line and the starting point image line, wherein the object distance is determined by multiplying the distance region width by the ratio of the reference point distance to the distance region image width and then adding the result to the starting point (19) of the visible distance region (15).

4. The method according to claim 1 or 2, wherein: The lighting device (5) and the optical sensor (7) are designed to operate in the near-infrared range.

5. The method according to claim 1 or 2, wherein: A plurality of photographs (23) are created which are ordered in time, wherein the temporal coordination of the lighting device (5) and the optical sensor (7) is varied in order to determine the change in the distance of the object (17) over time.

6. A control device (9) designed to carry out the method according to one of the preceding claims.

7. A distance measuring device (3) comprising an illumination device (5), an optical sensor (7) and a control device (9) according to claim 6.

8. A motor vehicle (1) comprising a distance measuring device (3) according to claim 7.

Images