DTOF lidar device with reduced optical crosstalk, and control method thereof

Abstract

A LiDAR device, including : an active light source configured to emit light to a scene; a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light; and circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

Description

[0001] DTOF LIDAR DEVICE WITH REDUCED OPTICAL CROSSTALK, AND CONTROL METHOD THEREOF

[0002] TECHNICAL FIELD

[0003] The present disclosure generally pertains to a LiDAR device, a device and a control method for a LiDAR device.

[0004] TECHNICAL BACKGROUND

[0005] Generally, direct time-of-flight (“dToF”) devices are known, which may also be referred to as LiDAR (“Light Detection and Ranging”) devices. Such dToF devices are typically used to determine distances to objects in a scene.

[0006] However, for example, when a LiDAR device illuminates a region of a scene that is bigger than a field-of-view of a single sensor element, a light detection pixel may receive a multipath interference (“MPI”) signal from other regions of the scene. It could be caused, for instance, by optical crosstalk in the receiver side optical system or by multiple scattering events of the illumination light at multiple objects in the scene, in particular, but not exclusively, when highly reflective objects are present in the scene.

[0007] The MPI signal may reduce the sensing accuracy and it is thus desirable to acquire depth data with at least a reduced amount of optical crosstalk or to at least measure or estimate or distinguish the optical crosstalk.

[0008] Although there exist techniques for LiDAR devices, it is generally desirable to improve the existing techniques.

[0009] SUMMARY

[0010] According to a first aspect, the disclosure provides a LiDAR device, comprising: an active light source configured to emit light to a scene; a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light; and circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different. According to a second aspect, the disclosure provides a device, comprising: a LiDAR device, including: an active light source configured to emit light to a scene, a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light, circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0011] According to a third aspect, the disclosure provides a control method for a LiDAR device, comprising: controlling an active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0012] Further aspects are set forth in the dependent claims, the drawings and the following description.

[0013] BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Embodiments are explained by way of example with respect to the accompanying drawings, in which:

[0015] Fig. 1 schematically illustrates an embodiment and an operating principle of a LiDAR device;

[0016] Fig. 2 schematically illustrates in a block diagram an embodiment of a data acquisition of a LiDAR sensor for a line scanning LiDAR device;

[0017] Fig. 3 schematically illustrates in a block diagram an example scene with an occurrence of optical crosstalk;

[0018] Fig. 4 schematically illustrates in a block diagram the example scene of Fig. 3;

[0019] Fig. 5 schematically illustrates examples of histograms acquired in a conventional time-of-flight measurement; Fig. 6 schematically illustrates in a block diagram an embodiment of a data acquisition of a LiDAR sensor for a line scanning LiDAR device;

[0020] Fig. 7 schematically illustrates examples of histograms acquired in an embodiment of a time-of- flight measurement;

[0021] Fig. 8 schematically illustrates examples of histograms acquired in an embodiment of a time-of- flight measurement;

[0022] Fig. 9 schematically illustrates examples of histograms acquired in an embodiment of a time-of- flight measurement;

[0023] Fig. 10 schematically illustrates in a flow diagram an embodiment of a control method for a LiDAR device; and

[0024] Fig. 11 schematically illustrates in a flow diagram an embodiment of a control method for a LiDAR device.

[0025] DETAILED DESCRIPTION OF EMBODIMENTS

[0026] Before a detailed description of the embodiments under reference of Fig. 3 is given, general explanations are made.

[0027] As mentioned in the outset, direct time-of-flight (“dToF”) devices are known, which may also be referred to as LiDAR (“Light Detection and Ranging”) devices. Such dToF devices are typically used to determine distances to objects in a scene.

[0028] For enhancing the general understanding of the present disclosure, an embodiment and an operating principle of a LiDAR device 1 is discussed in the following under reference of Fig. 1, which schematically illustrates the operating principle, and which may also apply to other embodiments of the present disclosure.

[0029] The LiDAR device 1 includes an active light source 2 and an imaging device 3 which are controlled by a controller (not shown), wherein the imaging device 3 includes a LiDAR sensor (not shown). The active light source 2 may be an addressable illuminator or emitter array (e.g., an addressable VCSEL (“Vertical-Cavity Surface-Emitting Laser”) array). In other embodiments, the active light source 2 may include at least two illuminators or emitters, which are individually addressable, placed next to each other.

[0030] The example LiDAR device 1 is based on the dToF technique and uses a Single-Photon Avalanche Diode (“SPAD”) array as the LiDAR sensor for its operation to acquire depth data, wherein the SPAD array includes a plurality of light detection pixels, wherein each light detection pixel includes one or more SPADs.

[0031] As depicted in Fig. 1, the dToF technique is based on a synchronized process of illuminating a scene 4 by the active light source 2 and of acquiring the reflected illumination light returning from the scene 4 by the imaging device 3.

[0032] The process starts, for example, with the emission of short light pulses toward the scene 4. When these light pulses interact with objects in the scene 4, a portion of the photons is thrown back towards the imaging device 3. The reflected illumination light is detected by the SPADs that have the probability of creating an avalanche current for each received photon.

[0033] Then, the LiDAR sensor records the arrival time for each light detection event with respect to the time of emission of a light pulse and groups them in discrete time intervals (bins) to create a histogram, thereby generating ToF data (histogram data).

[0034] In order to improve the signal-to-noise (“SNR”) ratio, this process may be repeated several times and the final histogram may thus be the sum of the histograms for each emitted light pulse.

[0035] Afterwards, the ToF data (histogram data) is processed to detect peaks in the histogram indicating the arrival time of the reflected pulse and, thus, a distance to an object in the scene (depth data).

[0036] Each light detection pixel in the LiDAR sensor may include one or more SPADs, and one histogram is built for each pixel (the light detection events of each SPAD of the light detection pixel may be summed) such that a 3D point cloud for the target field-of-view of the LiDAR system is obtained.

[0037] The LiDAR device 1 may, for example, be configured as a line scanning LiDAR device that emits a line of light to the scene 4 and moves the line of light along a predetermined direction (e.g., horizontal, vertical or diagonal) to scan the scene 4.

[0038] The LiDAR device 1 may, for example, also be configured as a spot LiDAR device that emits a plurality of light spots to the scene 4.

[0039] For further enhancing the general understanding of the present disclosure, an embodiment of a data acquisition of a LiDAR sensor for a line scanning LiDAR device is discussed in the following under reference of Fig. 1 and Fig. 2, which schematically illustrates the embodiment, and which may also apply to other embodiments of the present disclosure. In the case that the LiDAR device 1 of Fig. 1 is configured as a line scanning LiDAR, as mentioned above, the active light source 2 emits a line of light to the scene 4 which is moved over the scene 4 for scanning the scene 4.

[0040] When objects are present in the scene 4, a reflected line of light 61 may be incident on a LiDAR sensor 21 of the imaging device 3.

[0041] The LiDAR sensor 21 includes a plurality of light detection pixels 60 (e.g., including one or more SPADs) arranged, for example, in rows R-l,. . R-10 and columns. In each row R-l,. . ., R- 10 one or more adjacent light detection pixels may be activated (e.g., via a switch) to configure a macropixel 70 depending on a scanning position of the line of light or, in other words, depending on an currently illuminated part of the field-of-illumination of the active light source. Typically, a macropixel is configured by two or more adjacent light detection pixels, however, the present disclosure is not limited to the case of configuring macropixels in this way. In each row R-l,. . ., R-10 a single light detection pixel 60 may be activated in some embodiments.

[0042] The activated light detection pixels 60 perform photoelectric conversion on incident light to generate light detection events. The generated light detection events of each activated light detection pixel 60 of a macropixel 70 are binned together such that a single own histogram is generated for each macropixel 70.

[0043] The part of the reflected line of light 61 which is, for instance, incident on the macropixel 70 of row R-3 may cause scattering inside the imaging device 3 such that optical crosstalk may be incident, for example, on the macropixel 70 of row R-5 and R-7.

[0044] Returning to the general explanations, the optical crosstalk may lead to the generation of false positives, since in conventional LiDAR devices the MPI signal is mixed with the direct signal. When the optical crosstalk is accounted for by software processing, also false negatives may be possible in some cases.

[0045] In particular, reflections from highly reflective objects (which may be referred to as retroreflector or retroreflecting object, as generally known) are a source of optical crosstalk. The retroreflector itself may appear with a wrong dimension and the wrong dimensions may impact, for instance, the drivable area in automotive applications. Blooming effects may for example cause a LiDAR device or system to detect a retroreflective surface as larger than it is in reality. Examples of highly reflective objects may be road signs, road markings such as white or reflective paint, car license plates, headlights, rear lamps, mirrors, road studs, clothing or accessories, an illuminated screen or display, plastic reflectors on static or moving objects. Thus, the optical crosstalk (or MPI signal in general) may reduce the sensing accuracy of the LiDAR device such that it is thus desirable to acquire depth data with at least a reduced amount of optical crosstalk or to at least measure the optical crosstalk. Moreover, it is desirable to ensure that other objects within the optical crosstalk region are still detectable, in particular, when the other objects are present at the same distance as the retroflector.

[0046] It has been recognized that the optical crosstalk signal caused by a retroreflector can be separated in the histograms from the direct signal, when different parts of the scene are illuminated at different times.

[0047] Hence, some embodiments pertain to a LiDAR device, wherein the LiDAR device includes: an active light source configured to emit light to a scene; a LiDAR sensor including a plurality of light detection pixels, wherein each light detection pixel is configured to perform photoelectric conversion on incident light; and circuitry configured to control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of-illumination are different, and wherein the first time and the second time are different.

[0048] Some embodiments pertain to a device, wherein the device includes: a LiDAR device, including: an active light source configured to emit light to a scene, a LiDAR sensor including a plurality of light detection pixels, wherein each light detection pixel is configured to perform photoelectric conversion on incident light, circuitry configured to control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of-illumination are different, and wherein the first time and the second time are different.

[0049] The device may be a vehicle, a robot, a mobile electronic device (such as a smartphone, a tablet, a laptop, a head mounted display etc.), etc. and is not particularly limited. In some embodiments, the sensor is configured to generate signals in response to light detected at the plurality of light detection pixels and to provide the signals as control signals or for converting to control signals to an interface of the device. The device may use the signals or control signals to control a function of the device such as steering or breaking or providing user instruction.

[0050] The active light source includes a plurality of emitters, wherein each emitter may be a Light Emitting Diode (“LEDs”), a laser diode, a Vertical-Cavity Surface-Emitting Laser (“VCSELs”) or the like. Each emitter can be individually controlled to emit a light pulse and / or groups of emitters can be individually controlled to emit a light pulse. The active light source may include a plurality of drivers to drive each emitter or each group of emitters individually according to a respective control signal. An emitter may also be referred to as illuminator.

[0051] The LiDAR device is thus not limited to the emission of a first light pulse and a second light pulse with different emission timings to different parts of the field-of-illumination of the active light source, since the LiDAR device may emit a plurality of light pulses with mutually different emission timings to mutually different parts of the field-of-illumination.

[0052] The time difference between the first time and the second time may be constant or based on a random or pseudo-random number, which may also apply to embodiments with multiple light pulses for multiple different parts at multiple different emission timings.

[0053] In some embodiments, as mentioned above, the LiDAR device, i.e. the active light source of the LiDAR device, emits a light pulse to each of a plurality of mutually different (adjacent) parts of the field-of-illumination of the active light source with mutually different emission timings, wherein the time difference between the different emission timings is constant over a plurality of time-of-flight measurements, but may be different between adjacent parts of the field-of- illumination. For example, a first light pulse is emitted to a first part at a first time, a second light pulse is emitted to a second part adjacent too the first at a first time and a third light pulse is emitted to a third part adjacent to the second part at a third time, wherein the time difference between the first time and the second time may be different than the time difference between the second time and the third time.

[0054] In some of such embodiments, the active light source emits a line of light which is arranged vertically, and which is moved horizontally to scan a scene. In some of such embodiments, the vertically arranged line of light is divided into a plurality of parts, wherein each part is emitted by a different emitter of the active light source with a different emission timing. For the sake of illustration only, the line of light is divided into three parts, in some embodiments, such that the circuitry triggers the active light source to emit a first light pulse to an upper part, then after a constant delay the circuitry triggers the active light source to emit a second light pulse to a middle part, and then after a constant delay the circuitry triggers the active light source to emit a third light pulse to a lower part.

[0055] Thus, in some embodiments, the circuitry triggers illuminators with constant delays alternatively from top to bottom and from bottom to top.

[0056] The active light source may include optical parts such as lenses such as glass or plastic or liquid lenses, mirrors, optical filters etc.

[0057] The active light source may include mechanical parts to move the optical parts, e.g., piezo actuators.

[0058] In some embodiments, each light detection pixel includes one or more single-photon avalanche diodes, wherein each single-photon avalanche diode is configured to perform photoelectric conversion on incident light to generate light detection events.

[0059] The circuitry may be implemented as a separate controller of the LiDAR device or may be part of the active light source or may be part of the LiDAR sensor or a combination thereof.

[0060] The circuitry may include one or more processors. A processor may be or may include an application processor, a central processing unit (“CPU”), a graphical processing unit (“GPU”), a digital signal processor (“DSP”), a field-programmable gate array (“FPGA”), an application specific integrated circuit (“ASIC”) etc.

[0061] The circuitry may include one or more memory components, one or more input / output interfaces, one or more communication interfaces, one or more data bus interfaces and respective data busses to exchange data, one or more point-to-point connections to exchange data, etc.

[0062] The functionality of the circuitry may be implemented by typical electronic components configured to achieve the functionality as described herein. The functionality of the circuitry may be implemented in parts by typical electronic components and in parts by software configured to achieve the functionality as described herein. The functionality of the circuitry may be implemented by software configured to achieve the functionality as described herein.

[0063] A time-of-flight measurement includes at least a measurement time period during which a light pulse is emitted and at least a subset of the plurality of light detection pixels is activated to output light detection events. A time-of-flight measurement may include an output time period during which binned generated light detection events are output. In some embodiments, the time difference between the first time and the second time is larger than a pulse duration of at least one of the first and the second light pulse. Thus, a separation of optical crosstalk signal and direct signal may be enhanced.

[0064] In some embodiments, the first part of the field-of-illumination and the second part of the field- of-illumination are adjacent to each other. The optical crosstalk may typically appear on adjacent parts of the LiDAR sensor on which light from adjacent parts of the field-of-illumination is imaged.

[0065] In some embodiments, at least a part of the first light pulse that has been thrown back or reflected from the scene is incident on a first plurality of light detection pixels of the plurality of light detection pixels, and wherein at least a part of the second light pulse that has been thrown back or reflected from the scene causes optical crosstalk on the first plurality of light detection pixels, and wherein at least a part of the second light pulse that has been thrown back or reflected from the scene is incident on a second plurality of light detection pixels of the plurality of light detection pixels.

[0066] In some embodiments, the circuitry is further configured to bin generated light detection events with respect to the first time and the second time, respectively.

[0067] In some embodiments, light detection events generated by the first plurality of light detection pixels are binned with respect to the first time and light detection events generated by the second plurality of light detection pixels are binned with respect to the second time.

[0068] In some embodiments, the first time-of-flight measurement and the second time-of-flight measurement are the same time-of-flight measurement.

[0069] In such embodiments, the first pulse and the second pulse are emitted within the same measurement time period, however, at different time points with respect to the start of the measurement time period, wherein the time zero of the measurement time period may coincide with the first time.

[0070] Thus, the optical crosstalk signals and the direct signals may appear separated in the histograms.

[0071] In some embodiments, the first time-of-flight measurement and the second time-of-flight measurement are different time-of-flight measurements.

[0072] In such embodiments, the first pulse and the second pulse are emitted within different measurement time periods and at different time points with respect to the start of the respective measurement time period. Thus, the optical crosstalk signals and the direct signals are measured separately and may be processed separately.

[0073] In some embodiments, the circuitry is further configured to integrate light detection events generated during the first time-of-flight measurement and the second time-of-flight measurement.

[0074] In some embodiments, when the first and the second time-of-flight measurements are different time-of-flight measurements, integrating light detection events corresponds to summing up the light detection events generated during the first time-of-flight measurement and the second time- of-flight measurement.

[0075] Thus, in some embodiments, the circuitry is configured to generate a first histogram based on the light detection events generated during the first time-of-flight measurement and a second histogram based on the light detection events generated during the second time-of-flight measurement, wherein the circuitry is further configured to add the first histogram and the second histogram.

[0076] As mentioned above, generally, each of the first and second time-of-flight measurement may include multiple measurement time periods in which multiple first light pulses and multiple second light pulses are emitted, respectively, wherein in each measurement time period a histogram is generated which are summed up to generate the first histogram and the second histogram, respectively.

[0077] In some embodiments, when in those multiple first and second light pulses a real object appears at the same position in the histogram, the object signal may add up to one big peak. But, when the scattering is at different positions due to the time offsets, the scattering signal may not add up and may result in multiple small peaks in the histogram, which may be distinguished from the big peak from the object signal.

[0078] In some embodiments, the second time is based on a random number or pseudo random number.

[0079] In some embodiments, the circuitry is further configured to integrate light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of- flight measurements using the first time and a second plurality of time-of-flight measurements using the second time.

[0080] As mentioned above, in some embodiments, when the first and the second time-of-flight measurements are different time-of-flight measurements, integrating light detection events corresponds to summing up the light detection events generated during the first time-of-flight measurement and the second time-of-flight measurement.

[0081] Hence, for example, the optical crosstalk signal caused by the second light pulse may smear out in histograms of the first plurality of pixels, since the second light pulse is emitted at a random or pseudo random time with respect to the first time, which is fixed such that, due to the integration, the direct signal caused by the first light pulse builds up at the fixed position in time and the optical crosstalk signal is distributed in time without building up at a fixed position in time.

[0082] In some embodiments, the circuitry is further configured to detect whether a retroreflecting object is present in the scene and to use the second time in response to detecting presence of the retroreflecting object.

[0083] The detection of the retroreflecting object may be based on an appearance of a peak in a histogram, for example, the steepness of an edge, the relative intensity of the peak, the width of the peak, the area of the peak or the like.

[0084] Some embodiments pertain to a control method for a LiDAR device, wherein the control method includes controlling an active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0085] Some embodiments pertain to an information processing device, wherein the information processing device includes circuitry configured to: receive, from an interface of a LiDAR device, information representing light detection events integrated from first and second time-of-flight measurements; process the information representing the light detection events; and use the processed information to control a function of a vehicle.

[0086] The information processing device may be a computing device of the device as described herein, for example, the information processing device may be a computing device of a vehicle or a robot or the like. The information processing device may correspond to the device as described herein itself, for example, the information processing device may be a mobile electronic device. The information processing device may be a remote computer in some embodiments. In some embodiments, the function is at least one of steering, breaking and outputting an instruction to a driver to take control. In some of such embodiments, the information processing device is a computing device of a vehicle.

[0087] In some embodiments, the first and second time-of-flight measurements represent time-of-flight measurements in different illuminated fields of a captured scene, and wherein the circuitry is further configured to allow distinguishing of an overlapping echo of a retroreflective object in the first time-of-flight measurements from a second object in the second time-of-flight measurements.

[0088] In some embodiments, allowing the distinguishing is based on removing false positive peaks which are due to scattering light falsely indicating the presence of an object.

[0089] In some embodiments, allowing the distinguishing is based on avoiding false negatives by splitting signal peaks due to objects from signal peaks due to scattering light.

[0090] Some embodiments pertain to an information processing device, wherein the information processing device includes circuitry configured to: receive, from an interface of a LiDAR device, information representing light detection events from first and second time-of-flight measurements, the first and second time-of-flight measurements represent time-of-flight measurements in different illuminated fields of a captured scene; and process the information representing the light detection events to allow distinguishing of an overlapping echo of a retroreflective object in the first time-of-flight measurements from a second object in the second time-of-flight measurements.

[0091] The information processing device may be a computing device of the device as described herein, for example, the information processing device may be a computing device of a vehicle or a robot or the like. The information processing device may correspond to the device as described herein itself, for example, the information processing device may be a mobile electronic device. The information processing device may be a remote computer in some embodiments.

[0092] In some embodiments, the distinguishing is allowed due to integrating the light detection events of the first time-of-flight measurements and the second time-of-flight measurements, wherein the first time-of-flight measurements and the second time-of-flight measurements are different time- of-flight measurements.

[0093] In some embodiments, allowing the distinguishing is based on removing false positive peaks which are due to scattering light falsely indicating the presence of an object. In some embodiments, allowing the distinguishing is based on avoiding false negatives by splitting signal peaks due to objects from signal peaks due to scattering light.

[0094] The methods as described herein are also implemented in some embodiments as a computer program causing a computer and / or a processor to perform the method, when being carried out on the computer and / or processor. In some embodiments, also a non-transitory computer- readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.

[0095] Returning to Fig. 3, which schematically illustrates in a block diagram an example scene with an occurrence of optical crosstalk, which is discussed in the following under reference of Fig. 1, Fig. 2 and Fig. 3.

[0096] A line scanning LiDAR device 1 (not shown) - using the active light source 2 of Fig. 1 - illuminates a scene with a line of light 5 in which a high-reflective object 6, here a traffic sign, is present. A scanning direction of the line scanning LiDAR device 1 is perpendicular to the line of light 5. It will be understood that high-reflective object 6 may be another object not having intentionally reflective properties.

[0097] The scanning direction of the line scanning LiDAR device 1 may go from a first edge of the LiDAR sensor 21 to a second edge of the LiDAR sensor 21. In some embodiments, the scanning direction can be scanned from the first edge to the second edge of the LiDAR sensor 21 in a cyclic way. In some embodiments, the scanning direction can be scanned from the first edge to the second edge of the LiDAR sensor 21 and back to the first edge in a cyclic way.

[0098] The scene further includes a dark background above 8a and below 8b the high-reflective object 6 from which no illumination light is directly reflected. Of course, if any objects were present withing the field-of-view representing 8a and 8b, those objects may have some degree of reflection.

[0099] However, the illumination light directly reflected by object point 7 causes optical crosstalk inside the imaging device 3 of the LiDAR device 1. The optical (lens) crosstalk may be generated by diffraction effects, geometrical aberrations, and stray light in the imaging lens. Stray light may be caused by lens surface roughness and multiple reflections inside the lens elements. Diffraction gives a physical limit for the point spread function (“PSF”) of a lens and is defined by the lens f- number, and geometrical aberrations are caused by imperfections on the lens design and will also contribute to a finite PSF. The scene further includes a low-reflective object 10 with low reflectivity compared to the high- reflective object 6.

[0100] Due to the optical crosstalk caused by high-reflective object 6, an optical crosstalk signal is incident on such light detection pixels 60 which acquire light directly reflected from regions 9a and 9b.

[0101] For example, the optical crosstalk signal of the high-reflective object 6 overlaps with the light reflected by the low-reflective object 10 such that it may be difficult to discriminate a real direct echo from an optical crosstalk echo and it may be difficult to identify peaks due to optical crosstalk as not real objects. An autonomous driving system may therefore not detect a real object in a timely manner to take avoidance action or to indicate to a driver that they should take control in a timely manner due to such overlapping. The real object may only be detected and discriminated from an optical crosstalk echo once the vehicle gets closer to the real object, potentially making avoidance uncomfortable to passengers or more dangerous.

[0102] Fig. 4 schematically illustrates in a block diagram the example scene of Fig. 3, which is discussed in the following.

[0103] As depicted in Fig. 4, the scene is illuminated with a line of light 5 and the scene includes the high-reflective object 6 and the low-reflective object 10 at the same distance as the high- reflective object 6.

[0104] The scanned column is divided in a first part Pl, a second part P2 below the first part Pl, a third part P3 below the second part P2, a fourth part P4 below the third part P3 and a fifth part P5 below the fourth part P4.

[0105] Light that is directly thrown back from the different parts Pl to P5 are acquired by different light detection pixels 60 or macropixels 70 of the LiDAR sensor 21, as depicted in Fig. 2, wherein in each row R-l, . . . , R-10 one or more adjacent light detection pixels may be activated to configured a macropixel 70, as illustrated by the dotted boxes in Fig. 2. Typically, a macropixel is configured by two or more adjacent light detection pixels, however, the present disclosure is not limited to the case of configuring macropixels in this way. In each row R-l,. . ., R-10 a single light detection pixel 60 may be activated in some embodiments.

[0106] In a conventional time-of-flight measurement all parts of line of light 5 are emitted at the same time. Fig. 5 schematically illustrates examples of histograms acquired in a conventional time-of-flight measurement performed at the example scene of Fig. 3 and Fig. 4, which is discussed in the following under reference of Fig. 1, Fig. 2, Fig. 3 and Fig. 4.

[0107] The macropixel 70 of row R-l, for example, acquires light thrown back from the first part Pl of the example scene and generates a time sequence of light detection events which result in the depicted histogram H-l.

[0108] The macropixel 70 of row R-3, for example, acquires light thrown back from the second part P2 of the example scene and generates a time sequence of light detection events which result in the depicted histogram H-3.

[0109] The macropixel 70 of row R-5, for example, acquires light thrown back from the third part P3 of the example scene and generates a time sequence of light detection events which result in the depicted histogram H-5.

[0110] The macropixel 70 of row R-7, for example, acquires light thrown back from the fourth part P4 of the example scene and generates a time sequence of light detection events which result in the depicted histogram H-7.

[0111] The macropixel 70 of row R-9, for example, acquires light thrown back from the fifth part P5 of the example scene and generates a time sequence of light detection events which result in the depicted histogram H-9.

[0112] In the depicted histograms, the time tO marks the end of a previous measurement time period. Between time tO and time tl an output time period is present during which the binned light detection events generated during the previous measurement time period are output. At time tl the current measurement time period starts, the macropixels 70 are activated and the light detection events are binned with respect to the time tl . At time t2 the current measurement time period ends (macropixels 70 are deactivated), and the current output time period starts. At time t3 the current output time period ends, and the next measurement time period starts and so on.

[0113] The high-reflective object 6 is present in the second part P2 of the scene such that the histogram H-3 includes a sharp peak at the time tl + tobj, wherein tobj corresponds to the round-trip time of the emitted light pulse that is reflected at the high-reflective object 6 (direct signal) and acquired by the macropixel 70 of row R-3.

[0114] However, the direct signal incident on row R-3 causes optical crosstalk that is incident, e.g., on rows R-l, R-5, R-7 and R-9. For instance, the histogram H-l includes a peak at the time tl + tobj as well, although no object is present in the first part Pl of the scene which could reflect the emitted light pulse in this part of the scene. However, it may be difficult to discriminate whether it is a direct signal from a real object or whether it is an optical crosstalk signal.

[0115] The low-reflective object 10 is present in the fourth part P4 of the scene such that the histogram H-7 includes a peak at the time tl + tobj, wherein tobj corresponds to the round-trip time of the emitted light pulse that is reflected at the low-reflective object 10 (direct signal) and acquired by the macropixel 70 of row R-7, since the low-reflective object 10 is located at the same distance as the high-reflective object 6.

[0116] The direct signal incident on row R-7 causes optical crosstalk that is incident, e.g., only on neighboring rows R-5 and R-9, since the returning light amount is less for the low-reflective object 10 than for the high-reflective object 6.

[0117] Thus, the histogram H-5 includes a peak at the time tl + tobj as well due to optical crosstalk from the direct signal incident on rows R-3 and R-7, although no object is present in the third part P3 of the scene which could reflect the emitted light pulse in this part of the scene. However, it may be difficult to discriminate whether it is a real object or whether it is an optical crosstalk signal.

[0118] Also, the histogram H-9 includes a peak at the time tl + tobj as well, although no object is present in the fifth part P5 of the scene which could reflect the emitted light pulse in this part of the scene. However, it may be difficult to discriminate whether it is a real object or whether it is an optical crosstalk signal.

[0119] It has been recognized that the optical crosstalk signals and the direct signals can be separated in the histograms when different parts of the line of light 5 are emitted at different times, as will be discussed in the following.

[0120] Fig. 6 schematically illustrates in a block diagram an embodiment of a data acquisition of the LiDAR sensor 21 for the line scanning LiDAR device 1 of Fig. 1 and Fig. 3, which is discussed in the following under reference of Fig. 1, Fig. 3 and Fig. 6.

[0121] The active light source 2 emits a first light pulse to a first part of a field-of-illumination of the active light source 2 at a first time tl in a first time-of-flight measurement and emits a second light pulse to a second part of the field-of-illumination at a second time t4 in a second time-of- flight measurement, wherein the first part and the second part of the field-of-illumination are different, and wherein the first time tl and the second time t4 are different. When objects are present in the scene 4, a reflected line of light 61 may be incident on a LiDAR sensor 21 of the imaging device 3, which then includes a first partial line of light 61-1 corresponding to the first light pulse and a second partial line of light 61-2 corresponding to the second light pulse.

[0122] First macropixels 70 are activated at the first time tl and the binning of light detection events generated by the first macropixels 70 is performed with respect to the first time tl.

[0123] Second macropixels 71 are activated at the second time t4 and the binning of light detection events generated by the second macropixels 71 is performed with respect to the second time t4.

[0124] The first and the second time-of-flight measurement may be the same time-of-flight measurement, as will be discussed under reference of Fig. 7 in the following, which schematically illustrates examples of histograms acquired in an embodiment of a time-of-flight measurement.

[0125] In this embodiment, the scene of Fig. 4 is assumed. Two different parts of the line of light 5 are emitted at two different times to the scene, in particular, the part of the line of light 5 that covers the fourth part P4 and the fifth part P5 of the scene is emitted at the first time tl and the part of the line of light 5 that covers the first part Pl, the second part P2 and the third part P3 of the scene is emitted at the second time t4 which is a time amount tshift later than the first time tl.

[0126] The second macropixels 71 are activated at the second time t4 which corresponds to the time tl + tshift.

[0127] Then, the histogram H-3 includes the sharp peak from the high-reflective object 6 at the time t4 + tobj, since the second light pulse is emitted at the second time t4. However, the generated light detection events of the macropixel 71 of row R-3 are binned with respect to t4 such that the correct distance to the high-reflective object 6 is obtained based on the round-trip time tobj.

[0128] The first macropixels 70 are activated at the first time tl.

[0129] Then, the histogram H-7 includes a peak at the time tl + tobj from the low-reflective object 10, since the first light pulse is emitted at the first time tl .

[0130] Moreover, the histogram H-7includes an optical crosstalk peak at time tl + tobj + tshift which is caused by the direct signal incident on row R-3.

[0131] Hence, the peak from the low-reflective object 10 and the peak from the optical crosstalk are separated by the applied time shift with the time amount tshift and, thus, the low-reflective object 10 is detectable. It has been recognized that the different emission times for different parts of the scene may be applied in different time-of-flight measurements such that the optical crosstalk signals and the direct signals are measured separately and may be processed separately.

[0132] This will be discussed under reference of Fig. 8 and Fig. 9, which schematically illustrate examples of histograms acquired in an embodiment of time-of-flight measurements.

[0133] In Fig. 8 example histograms H-l to H-9 are depicted for the first time-of-flight measurement in which only the first light pulse is emitted to the fourth part P4 and the fifth part P5 of the scene of Fig. 4.

[0134] As a result, the histograms H-7 and H-9 include the peak at tl + tobj from the low-reflective object 10 but not the optical crosstalk signal due to the high-reflective object 6.

[0135] The histograms H-l, H-3 and H-5 only include ambient light contributions, in particular, the histogram H-5 does not include the optical crosstalk due to the direct signal incident on row R-7, since the second macropixels 71 are activated later at the second time t4.

[0136] In Fig. 9 example histograms H-l to H-9 are depicted for the second time-of-flight measurement in which only the second light pulse is emitted to the first part Pl, the second part P2 and the third part P3 of the scene of Fig. 4.

[0137] As a result, the histogram H-3 includes the sharp peak at time t4 + tobj from the high-reflective object 6.

[0138] The histograms H-l and H-5 include an optical crosstalk signal at time t4 + tobj caused by the direct signal that is incident ono row R-3.

[0139] The histograms H-7 and H-9 include the optical crosstalk signal at time tl + tobj + tshift but not the direct signal from the low-reflective object 10.

[0140] Fig. 10 schematically illustrates in a flow diagram an embodiment of a control method 120 for a LiDAR device, which is discussed in the following.

[0141] The control method 120 may be performed by the circuitry as described herein.

[0142] At 121, an active light source is controlled to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different, as discussed herein. At 122, generated light detection events are binned with respect to the first time and the second time, respectively, as discussed herein.

[0143] Fig. 11 schematically illustrates in a flow diagram an embodiment of a control method 150 for a LiDAR device, which is discussed in the following.

[0144] The control method 150 may be performed by the circuitry as described herein.

[0145] At 151, an active light source is controlled to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different, as discussed herein.

[0146] At 152, light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of-flight measurements using the first time and a second plurality of time-of-flight measurements using the second time are integrated, wherein the second time is based on a random number, as discussed herein.

[0147] It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding.

[0148] All circuitry described in this specification and claimed in the appended claims can, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such circuitry can, if not stated otherwise, be implemented by software.

[0149] In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be appreciated that a computer program providing such software control and a transmission, storage or other medium by which such a computer program is provided are envisaged as aspects of the present disclosure.

[0150] Note that the present technology can also be configured as described below.

[0151] (1) A LiDAR device, wherein the LiDAR device includes: an active light source configured to emit light to a scene; a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light; and circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0152] (2) The LiDAR device of (1), wherein the first time-of-flight measurement and the second time-of-flight measurement are the same time-of-light measurement.

[0153] (3) The LiDAR device of (1), wherein the first time-of-flight measurement and the second time-of-flight measurement are different time-of-flight measurements.

[0154] (4) The LiDAR device of anyone of (1) to (3), wherein the circuitry is further configured to integrate light detection events generated during the first time-of-flight measurement and the second time-of-flight measurement.

[0155] (5) The LiDAR device of anyone of (1) to (4), wherein the second time is based on a random number or pseudo random number.

[0156] (6) The LiDAR device of anyone of (1) to (5), wherein the circuitry is further configured to integrate light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of-flight measurements using the first time and a second plurality of time-of-flight measurements using the second time.

[0157] (7) The LiDAR device of anyone of (1) to (6), wherein the circuitry is further configured to detect whether a retroreflecting object is present in the scene and to use the second time in response to detecting presence of the retroreflecting object.

[0158] (8) The LiDAR device of anyone of (1) to (7), wherein the time difference between the first time and the second time is larger than a pulse duration of at least one of the first and the second light pulse.

[0159] (9) The LiDAR device of anyone of (1) to (8), wherein each light detection pixel includes a single-photon avalanche diode.

[0160] (10) The circuitry of anyone of (1) to (9), wherein the circuitry is further configured to bin generated light detection events with respect to the first time and the second time, respectively.

[0161] (11) A device, wherein the device includes: a LiDAR device, including: an active light source configured to emit light to a scene, a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light, circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0162] (12) A control method for a LiDAR device, wherein the control method includes: controlling an active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

[0163] (13) The control method of (12), wherein the first time-of-flight measurement and the second time-of-flight measurement are the same time-of-light measurement.

[0164] (14) The control method of (12), wherein the first time-of-flight measurement and the second time-of-flight measurement are different time-of-flight measurements.

[0165] (15) The control method of anyone of (12) to (14), further including integrating light detection events generated during the first time-of-flight measurement and the second time-of-flight measurement.

[0166] (16) The control method of anyone of (12) to (15), wherein the second time is based on a random number or pseudo random number.

[0167] (17) The control method of anyone of (12) to (16), further including integrating light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of-flight measurements using the first time and a second plurality of time-of-flight measurements using the second time.

[0168] (18) The control method of anyone of (12) to (17), further including detecting whether a retroreflecting object is present in the scene and to use the second time in response to detecting presence of the retroreflecting object. (19) The control method of anyone of (12) to (18), wherein the time difference between the first time and the second time is larger than a pulse duration of at least one of the first and the second light pulse.

[0169] (20) The control method of anyone of (12) to (19), further including binning generated light detection events with respect to the first time and the second time, respectively.

[0170] (21) A computer program comprising program code causing a computer to perform the control method according to anyone of (12) to (20), when being carried out on a computer.

[0171] (22) A non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the control method according to anyone of (12) to (20) to be performed.

[0172] (23) An information processing device, including circuitry configured to: receive, from an interface of a LiDAR device, information representing light detection events integrated from first and second time-of-flight measurements; process the information representing the light detection events; and use the processed information to control a function of a vehicle.

[0173] (24) The information processing device of (23), wherein the function is at least one of steering, breaking and outputting an instruction to a driver to take control.

[0174] (25) The information processing of (23) or (24), wherein the first and second time-of-flight measurements represent time-of-flight measurements in different illuminated fields of a captured scene, and wherein the circuitry is further configured to allow distinguishing of an overlapping echo of a retror effective object in the first time-of-flight measurements from a second object in the second time-of-flight measurements.

[0175] (26) The information processing device of (25), wherein allowing the distinguishing is based on removing false positive peaks which are due to scattering light falsely indicating the presence of an object.

[0176] (27) The information processing device of (25) or (26), wherein allowing the distinguishing is based on avoiding false negatives by splitting signal peaks due to objects from signal peaks due to scattering light.

[0177] (28) An information processing device, including circuitry configured to: receive, from an interface of a LiDAR device, information representing light detection events from first and second time-of-flight measurements, the first and second time-of-flight measurements represent time-of-flight measurements in different illuminated fields of a captured scene; and process the information representing the light detection events to allow distinguishing of an overlapping echo of a retror effective object in the first time-of-flight measurements from a second object in the second time-of-flight measurements.

[0178] (29) The information processing device of (26), wherein the distinguishing is allowed due to integrating the light detection events of the first time-of-flight measurements and the second time-of-flight measurements, wherein the first time-of-flight measurements and the second time- of-flight measurements are different time-of-flight measurements. (30) The information processing device of (28) or (29), wherein allowing the distinguishing is based on removing false positive peaks which are due to scattering light falsely indicating the presence of an object.

[0179] (31) The information processing device of anyone of (28) to (30), wherein allowing the distinguishing is based on avoiding false negatives by splitting signal peaks due to objects from signal peaks due to scattering light.

Claims

CLAIMS1. A LiDAR device, comprising: an active light source configured to emit light to a scene; a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light; and circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

2. The LiDAR device of claim 1, wherein the first time-of-flight measurement and the second time-of-flight measurement are the same time-of-light measurement.

3. The LiDAR device of claim 1, wherein the first time-of-flight measurement and the second time-of-flight measurement are different time-of-flight measurements.

4. The LiDAR device of claim 1, wherein the circuitry is further configured to integrate light detection events generated during the first time-of-flight measurement and the second time- of-flight measurement.

5. The LiDAR device of claim 1, wherein the second time is based on a random number or pseudo random number.

6. The LiDAR device of claim 1, wherein the circuitry is further configured to integrate light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of-flight measurements using the first time and a second plurality of time- of-flight measurements using the second time.

7. The LiDAR device of claim 1, wherein the circuitry is further configured to detect whether a retroreflecting object is present in the scene and to use the second time in response to detecting presence of the retroreflecting object.

8. The LiDAR device of claim 1, wherein the time difference between the first time and the second time is larger than a pulse duration of at least one of the first and the second light pulse.

9. The LiDAR device of claim 1, wherein each light detection pixel includes a single-photon avalanche diode.

10. The circuitry of claim 1, wherein the circuitry is further configured to bin generated light detection events in accordance with the first time and the second time, respectively.

11. A device, comprising: a LiDAR device, including: an active light source configured to emit light to a scene, a LiDAR sensor including a plurality of light detection pixels, each light detection pixel being configured to perform photoelectric conversion on incident light, circuitry configured to: control the active light source to emit a first light pulse to a first part of a field-of-illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

12. A control method for a LiDAR device, comprising: controlling an active light source to emit a first light pulse to a first part of a field-of- illumination of the active light source at a first time in a first time-of-flight measurement and to emit a second light pulse to a second part of the field-of-illumination at a second time in a second time-of-flight measurement, wherein the first part and the second part of the field-of- illumination are different, and wherein the first time and the second time are different.

13. The control method of claim 12, wherein the first time-of-flight measurement and the second time-of-flight measurement are the same time-of-light measurement.

14. The control method of claim 12, wherein the first time-of-flight measurement and the second time-of-flight measurement are different time-of-flight measurements.

15. The control method of claim 12, further comprising integrating light detection events generated during the first time-of-flight measurement and the second time-of-flight measurement.

16. The control method of claim 12, wherein the second time is based on a random number or pseudo random number.

17. The control method of claim 12, further comprising integrating light detection events generated during a plurality of time-of-flight measurements including a first plurality of time-of- flight measurements using the first time and a second plurality of time-of-flight measurements using the second time.

18. The control method of claim 12, further comprising detecting whether a retroreflecting object is present in the scene and to use the second time in response to detecting presence of the retroreflecting object.

19. The control method of claim 12, wherein the time difference between the first time and the second time is larger than a pulse duration of at least one of the first and the second light pulse.

20. The control method of claim 12, further comprising binning generated light detection events with respect to the first time and the second time, respectively.

Images