Lidar sensor, lidar system and methods

Abstract

A LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

Description

[0001] LIDAR SENSOR, LIDAR SYSTEM AND METHODS

[0002] TECHNICAL FIELD

[0003] The present disclosure generally pertains to a LiDAR sensor, a LiDAR system, a method for a LiDAR sensor and a method for a LiDAR system.

[0004] TECHNICAL BACKGROUND

[0005] Generally, direct time-of-flight (“dToF”) systems are known, which may also be referred to as LiDAR (“Light Detection and Ranging”). Such dToF systems are typically used to determine distances to objects in a scene.

[0006] However, for example, when a LiDAR system illuminates a region of a scene that is bigger than a field-of-view of a single sensor element, a light detection pixel may receive a multipath interference (“MPT’) signal from other regions of the scene. It could be caused, for instance, by optical crosstalk in the receiver side optical system or by multiple scattering events of the illumination light at multiple objects in the scene. The MPI signal may reduce the sensing accuracy and it is thus desirable to mitigate the effect of the MPI signal on the determined distances.

[0007] From the International patent application publication WO 2023 / 041465 Al, a method for mitigating the effect of the MPI signal is known by using first and second binning of light detection pixels. Some embodiments of the present disclosure may avoid frame rate limitations, potentially allowing operation at a higher frame rate and / or may improve power efficiency in some cases and may avoid and / or limit a need for post-processing of the acquired signals and histograms.

[0008] Some embodiments of the disclosure may mitigate the effect of the MPI signal by a fast compensation of it on-chip for allowing a framerate to be increased and for allowing a system complexity to be reduced.

[0009] Although there exist techniques for LiDAR systems, it is generally desirable to improve the existing techniques.

[0010] SUMMARY

[0011] According to a first aspect, the disclosure provides a LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0012] According to a second aspect, the disclosure provides a LiDAR system, comprising: an illumination device configured to illuminate a scene with modulated light; an imaging unit configured to image reflected illumination light on a LiDAR sensor; and the LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident, acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0013] According to a third aspect, the disclosure provides a method for a LiDAR sensor, comprising: acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0014] According to a fourth aspect, the disclosure provides a method for a LiDAR system, comprising: illuminating a scene with modulated light; imaging reflected illumination light on a LiDAR sensor; acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0015] Further aspects are set forth in the dependent claims, the drawings and the following description.

[0016] BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Embodiments are explained by way of example with respect to the accompanying drawings, in which:

[0018] Fig. 1 schematically illustrates an operating principle of a LiDAR system;

[0019] Fig. 2 schematically illustrates an example of an occurrence of a multipath interference signal;

[0020] Fig. 3 schematically illustrates in a block diagram an embodiment of an imaging device in which indirectly reflected illumination light is caused by crosstalk of directly reflected illumination light;

[0021] Fig. 4 schematically illustrates in a block diagram an embodiment of a Single-Photon Avalanche Diode-based LiDAR sensor in which intra-sensor crosstalk contributes to a multipath interference signal;

[0022] Fig. 5 schematically illustrates in a block diagram an embodiment of a LiDAR sensor;

[0023] Fig. 6 schematically illustrates in a block diagram an embodiment of a line scanning LiDAR system;

[0024] Fig. 7 schematically illustrates in a block diagram in Fig. 7A an embodiment of a data acquisition of a LiDAR sensor of a line scanning LiDAR system at a first scanning position and in Fig. 7B an embodiment of a data acquisition of a LiDAR sensor of a line scanning LiDAR system at a second scanning position;

[0025] Fig. 8 schematically illustrates an embodiment of a light detection pixel;

[0026] Fig. 9 schematically illustrates an embodiment of a direct macropixel and an embodiment of a global macropixel;

[0027] Fig. 10 schematically illustrates in a block diagram an embodiment of a LiDAR sensor;

[0028] Fig. 11 schematically illustrates in a block diagram an embodiment of a spot LiDAR system; Fig. 12 schematically illustrates in a block diagram an embodiment of a data acquisition of a LiDAR sensor of a spot LiDAR system; and

[0029] Fig. 13 schematically illustrates in a flow diagram a method for a LiDAR system.

[0030] DETAILED DESCRIPTION OF EMBODIMENTS

[0031] Before a detailed description of the embodiments under reference of Fig. 6 is given, general explanations are made.

[0032] As mentioned in the outset, generally, direct time-of-flight (“dToF”) systems are known, which may also be referred to as LiDAR (“Light Detection and Ranging”). Such dToF systems are typically used to determine distances to objects in a scene.

[0033] For enhancing the general understanding of the present disclosure, an operating principle of a LiDAR system 1 is discussed in the following under reference of Fig. 1, which schematically illustrates the operating principle, and which may also apply to other embodiments of the present disclosure.

[0034] The LiDAR system 1 includes an illumination device 2 and an imaging device 3 which are controlled by a control (not shown), wherein the imaging device 3 includes a LiDAR sensor (not shown).

[0035] The example LiDAR system 1 is based on the dToF technique and uses on a Single-Photon Avalanche Diode (“SPAD”) array as the LiDAR sensor for its operation to acquire depth data.

[0036] As depicted in Fig. 1, the dToF technique is based on a synchronized process of illuminating a scene 4 by the illumination device 2 and of detecting the reflected illumination light returning from the scene 4 by the imaging device 3.

[0037] The process starts, for example, with the emission of short light pulses toward the scene 4. When these light pulses interact with objects in the scene 4, a portion of the photons is reflected towards the imaging device 3. The reflected illumination light is detected by the SPADs that have the probability of creating an avalanche current for each received photon.

[0038] Then, the LiDAR sensor records the arrival time for each light detection event and groups them in discrete time intervals (bins) to create a histogram, thereby generating ToF data (histogram data).

[0039] In order to improve the signal-to-noise (“SNR”) ratio, this process may be repeated several times and the final histogram may thus be the sum of the histograms for each emitted light pulse. Afterwards, the ToF data (histogram data) is processed to detect peaks in the histogram indicating the arrival time of the reflected pulse and, thus, a distance to an object in the scene (depth data).

[0040] Each light detection pixel in the LiDAR sensor may include one or more SPADs, and one histogram is built for each pixel (the light detection events of each SPAD of the light detection pixel may be summed) such that a 3D point cloud for the target field-of-view of the LiDAR system is obtained.

[0041] Returning to the general explanations, however, as further mentioned in the outset, when a LiDAR system illuminates a region of a scene that is bigger than a field-of-view of a single sensor element, a light detection pixel may receive a multipath interference (“MPT’) signal from other regions of the scene. It could be caused, for example, by optical crosstalk in the receiver side optical system or by multiple scattering of the illumination light at objects in the scene.

[0042] As the LiDAR system typically acquires ToF data concurrently for the entire field-of-view, or portions of it, in a non-ideal system the reflected illumination light from different points in the scene may influence each other. This may result in histograms showing direct peaks that belong to the target region of the scene, and indirect peaks that belong to other parts of the scene, which, however, should not be in the histogram if the system was ideal.

[0043] This MPI signal may reduce the sensing accuracy and it is thus desirable to mitigate the effect of the MPI signal on the determined distances.

[0044] For further enhancing the general understanding of the present disclosure, an example of an occurrence of a MPI signal is schematically illustrated in Fig. 2, which is discussed in the following.

[0045] A line scanning LiDAR system (not shown) illuminates a scene with a line of light 5 in which a highly reflective object 6, here a traffic sign, is present. A scanning direction of the line scanning LiDAR system is perpendicular to the line of light 5. It will be understood that object 6 may be another object not having intentionally reflective properties.

[0046] A first light detection pixel of a LiDAR sensor of the line scanning LiDAR system acquires a direct signal contribution resulting from illumination light directly reflected at object point 7.

[0047] The scene further includes a dark background above 8a and below 8b the object 6 from which no illumination light is directly reflected. Of course, if any objects were present withing the field of view representing 8a and 8b, those objects may have some degree of reflection. However, the illumination light directly reflected by object point 7 causes optical crosstalk inside an imaging unit of the LiDAR system. The optical (lens) crosstalk may be generated by diffraction effects, geometrical aberrations, and stray light in the imaging lens. Stray light may be caused by lens surface roughness and multiple reflections inside the lens elements. Diffraction gives a physical limit for the point spread function (“PSF”) of a lens and is defined by the lens flnumber, and geometrical aberrations are caused by imperfections on the lens design and will also contribute to a finite PSF.

[0048] Due to the optical crosstalk, indirectly reflected illumination light is incident on a second light detection pixel in the vicinity of the first light detection pixel. Thus, on the second light detection pixel a MPI signal contribution is present.

[0049] The imaging unit of the LiDAR system would image directly reflected illumination light from a scene region 9 on the second light detection pixel. As there is a dark background in scene region 9, the second light detection pixel does not acquire a direct signal contribution, however, as discussed above, a MPI signal contribution.

[0050] Hence, the histogram of the second light detection pixel indicates an alleged object point in the scene region 9, which is however not present. Accordingly, the object 6 appears to be larger in the vertical direction than it actually is, which may induce an incorrect decision on an advanced driver assistance system (“ADAS”) consuming the depth data from the LiDAR system.

[0051] Apart from this, indirectly reflected illumination light may further be incident on the first light detection pixel, for example, caused by crosstalk or scattering of direct reflected illumination light from an object point of the object 6, which is above or below the object point 7, and which is also illuminated by the line of light 5. Moreover, this indirectly reflected illumination may further be incident on the second light detection pixel.

[0052] For further enhancing the general understanding of the present disclosure, an embodiment of an imaging device 12 in which indirectly reflected illumination light is caused by crosstalk of directly reflected illumination light is discussed in the following under reference of Fig. 3, which schematically illustrates the embodiment in a block diagram.

[0053] The imaging device 12 includes an imaging unit 20 and a LiDAR sensor 21, wherein the LiDAR sensor 21 is arranged in a focal plane of the imaging unit for the sake of illustrating the optical crosstalk and without limiting the present disclosure in this regard. The distance between the imaging unit 20 and the LiDAR sensor 21 may be adjustable and the optical properties of the imaging unit 20 may be adjustable, e.g., by lens movement or by using liquid lenses. The imaging unit 20 includes a first lens 30, a second lens 31 and a third lens 32 which image light rays 40a and 40b that pass an aperture (not shown) on the LiDAR sensor 21.

[0054] While traveling through the lens stack, the light rays 40a and 40b are reflected at the lens surfaces, for example, at surface (e). The reflected light rays 41a and 41b are reflected again, for example, at surface (b) such that they are incident on the LiDAR sensor 21 at a different position than the light rays 40a and 40b.

[0055] Hence, the directly reflected illumination light for a first light detection pixel of the LiDAR sensor 21 causes indirectly reflected illumination light to be incident on a second light detection pixel of the LiDAR sensor 21 by optical crosstalk.

[0056] For further enhancing the general understanding of the present disclosure, an embodiment of a SPAD-based LiDAR sensor 21 in which intra-sensor crosstalk contributes to a MPI signal is discussed under reference of Fig. 4, which schematically illustrates the embodiment in a block diagram, and which may also apply to other embodiments of the present disclosure.

[0057] When light is incident on the SPAD 50, a photo-generated electron 51 may be present and may diffuse into a depletion region 52 of the SPAD 50.

[0058] Once the photo-generated electron 51 reaches the depletion region 52, an avalanche process may occur in which further electrons 53 and 55 may be generated, as generally known.

[0059] Typically, these electrons 53 and 55 would be driven towards the cathode causing a voltage drop at the cathode and an avalanche signal to be output by the SPAD 50 until the cathodic voltage drops below the breakdown voltage and the bias voltage is restored again at the cathode.

[0060] However, the electrons 53 and 55 may cause intra-sensor crosstalk, since during the avalanche process photons are emitted due to electroluminescence, and a portion of these photons may reach a neighboring SPAD by a direct or indirect path.

[0061] The indirect path usually involves at least one reflection at the bottom layer of the SPAD 50, where a resonance peak can be found for the total internal reflection condition, which depends on the difference of index of refraction of the layers and the incidence angle of the generated photon.

[0062] The direct path may lead to the photo-generation of an electron 54 in the neighboring SPAD, where it may diffuse into the depletion region and initiate an avalanche process.

[0063] Moreover, the electrons 53 and 55 may directly electrically diffuse into the neighboring SPAD, where they may diffuse into the depletion region and initiate an avalanche process. Returning to the general explanations, it has been recognized that the effect of the MPI signal should be mitigated, as will be discussed in the following.

[0064] As mentioned in the outset from the International patent application publication WO 2023 / 041465 Al, a method for mitigating the effect of the MPI signal is known by using first and second binning of light detection pixels. Some embodiments of the present disclosure may avoid frame rate limitations, potentially allowing operation at a higher frame rate and / or may improve power efficiency in some cases and may avoid and / or limit a need for post-processing of the acquired signals and histograms.

[0065] It is thus an object to mitigate the effect of the MPI signal by a fast compensation of it on-chip for allowing a framerate to be increased and for allowing a system complexity to be reduced.

[0066] Hence, some embodiments pertain to a LiDAR system, wherein the LiDAR system includes: an illumination device configured to illuminate a scene with modulated light; an imaging unit configured to image reflected illumination light on a LiDAR sensor; and the LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident, acquire a MPI signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the MPI signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0067] Accordingly, some embodiments pertain to a LiDAR sensor, wherein the LiDAR sensor is configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0068] The illumination device may include an illuminator which may include one or more light sources such as one or more Light Emitting Diodes (“LEDs”), one or more laser diodes, one or more Vertical-Cavity Surface-Emitting Lasers (“VCSELs”) etc. The illuminator may include one or more drivers to drive each light source individually according to a respective control signal or to drive all light sources according to a control signal for emitting modulated light.

[0069] The illumination device may include optical parts such as lenses such as glass or plastic or liquid lenses, mirrors, optical filters etc.

[0070] The illumination device may include mechanical parts to move the optical parts, e.g., piezo actuators.

[0071] The modulated light may be temporally and spatially modulated such that the modulated light includes, for instance, a train of light pulses emitted as a line of light or as a plurality of light spots or as a spatial Gaussian profile (which may also be referred to as flooded illumination) or the like.

[0072] The imaging unit may include optical parts such as lenses such as glass or plastic or liquid lenses, mirrors, optical filters etc.

[0073] The imaging unit may include mechanical parts to move the optical parts, e.g., piezo actuators.

[0074] The LiDAR system may further include a control configured to control the overall operation of the LiDAR system including, for example, light emission timing, data acquisition, synchronization between the illumination device and the LiDAR sensor etc., as generally known.

[0075] The LiDAR sensor includes a plurality of light detection pixels which may be arranged in a row or in an array in rows and columns or the like.

[0076] In some embodiments, each light detection pixel includes a SPAD. Each SPAD is configured to generate a light detection event in response to incident light, as generally known.

[0077] Each light detection pixel of a part of the plurality of light detection pixels of the LiDAR sensor receives directly reflected illumination light from a different predetermined region of the scene, which is illuminated, and which is within the field-of-view of the respective light detection pixel (assuming a reflective object is present in the predetermined region of the scene).

[0078] Thus, the LiDAR sensor is configured to acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, for example, based on an arrival time of the directly reflected illumination light with respect to an emission start timing of the modulated light.

[0079] The LiDAR sensor acquires the direct signal contribution with a direct macropixel. In some embodiments, the direct macropixel corresponds to a plurality of adjacent light detection pixels of the plurality of light detection pixels. Generated light detection events of the individual light detection pixels are binned.

[0080] The region of interest inside the actively illuminated part of the LiDAR sensor may be any part of the LiDAR sensor on which directly reflected illumination light is present. The region of interest may depend, for example, on the scanning position of the LiDAR system.

[0081] However, as discussed above, the direct macropixel may further acquire a MPI signal contribution, which may reduce a sensing accuracy.

[0082] Hence, the LiDAR sensor acquires a MPI signal contribution with a different macropixel, in particular, with a global macropixel which does not receive directly reflected illumination light, since its field-of-view is never directly illuminated by the LiDAR system.

[0083] Therefore, the MPI signal contribution acquired with the global macropixel is used to estimate the MPI signal contribution acquired with the direct macropixel for mitigating the effect of MPI on the sensing accuracy.

[0084] In some embodiments, the global macropixel corresponds to a plurality of adjacent light detection pixels of the plurality of light detection pixels. Generated light detection events of the individual light detection pixels are binned.

[0085] In some embodiments, the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, in particular, wherein the multipath interference signal contribution further includes a contribution from intra-sensor crosstalk, as discussed above.

[0086] For further enhancing the general understanding of the present disclosure, an embodiment of LiDAR sensor 200 is discussed under reference of Fig. 5, which schematically illustrates the embodiment in a block diagram, and which may also apply to other embodiments of the present disclosure.

[0087] The LiDAR sensor 200 has a part 201 that is actively illuminated. In other words, the part 201 receives directly reflected illumination light, while the rest of the LiDAR sensor - the part of LiDAR sensor which is outside the part 201 that is actively illuminated - only receives indirectly reflected illumination light.

[0088] Hence, a global macropixel 202 is configured outside the part 201 that is actively illuminated, since this part allows to acquire the MPI signal contribution. Returning to the general explanations, for allowing a framerate to be increased, in some embodiments, the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

[0089] In some embodiments, the LiDAR sensor includes a counter circuitry configured to be incremented by generated light detection events of the direct macropixel and to be decremented by generated light detection events of the global macropixel.

[0090] This allows a fast compensation of the MPI signal contribution directly on-chip for allowing a framerate to be increased and for allowing a system complexity to be reduced.

[0091] A decrease in the MPI signal contribution acquired with the direct macropixel is expected, since the MPI signal contribution of the global macropixel is subtracted directly on-chip.

[0092] Moreover, when the global macropixel acquires a MPI signal contribution that is not acquired with the direct macropixel and which does not overlap in time with the acquired direct signal contribution, the counter circuitry simply counts down to zero for the respective bin such that the resulting histogram is not negatively influenced.

[0093] Moreover, even if the MPI signal contribution acquired with the global macropixel is not acquired with the direct macropixel and overlaps in time with the acquired direct signal contribution, the influence on the peak detection is expected to be small, since the light amount of the directly reflected illumination light is typically much higher than the light amount of the indirectly reflected illumination light.

[0094] Hence, the peak in the histogram corresponding to the acquired direct signal contribution would only be slightly reduced and, thus, the peak detection is not largely effected.

[0095] In some embodiments, the LiDAR sensor is further configured to update a histogram based on an output of the counter circuitry.

[0096] Thus, no or less signal post-processing may be required, in some cases, thereby reducing system complexity.

[0097] In some embodiments, the counter circuitry is further configured to be resetted for each new bin of the histogram.

[0098] Thus, no time-to-digital converters (“TDC”) may be required, thereby reducing system complexity.

[0099] Similarly, a plurality of different direct macropixel may be used to acquire a plurality of different direct signal contributions. Hence, in some embodiments, the LiDAR sensor includes a plurality of counter circuitries, each configured to be incremented by generated light detection events of an associated direct macropixel and to be decremented by generated light detection events of the global macropixel, wherein the LiDAR sensor is further configured to simultaneously acquire a plurality of direct signal contributions, each with a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitries.

[0100] Some embodiments pertain to a method for a LiDAR sensor, including: acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0101] The method may be performed by the LiDAR sensor as described herein.

[0102] Some embodiments pertain to a method for a LiDAR system, including: illuminating a scene with modulated light; imaging reflected illumination light on a LiDAR sensor; acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0103] The method may be performed by the LiDAR system as described herein.

[0104] Any LiDAR sensor discussed herein may also be referred to as a light receiver or as a light detection sensor.

[0105] Returning to Fig. 6, which schematically illustrates in a block diagram an embodiment of a line scanning LiDAR system 10, which is discussed in the following under reference of Figs. 6 to 10. The line scanning LiDAR system 10 includes an illumination device 11, an imaging device 12 (e.g., the imaging device 12 of Fig. 3 without limiting the disclosure to this embodiment) and a control 13.

[0106] The control 13 basically controls the overall operation of the line scanning LiDAR system 10, for example, light emission timing, data acquisition, synchronization between the illumination device 11 and the imaging device 12, etc., as generally known.

[0107] The illumination device 11 includes a light source 14 and a scanning unit 15.

[0108] The imaging device 12 includes an imaging unit 20 and a LiDAR sensor 21, wherein the LiDAR sensor 21 includes a plurality of light detection pixels, each light detection pixel including a SPAD (e.g., the SPAD of Fig. 4 without limiting the disclosure to this embodiment).

[0109] The light source 14 includes a laser diode configured to emit a line of light 5 (e.g., ID illumination light pattern) within a field-of-illumination 16 towards a scene 17 in which an object 19 is present.

[0110] Any light coming from the scene 17 is collected by the imaging device 12, when it is within a field-of-view 22 of the imaging device 12.

[0111] The illumination device 11 only illuminates a part 18 of the field-of-view 22 of the imaging device 12 such that the LiDAR sensor 21 includes light detection pixel regions on which no directly reflected illumination light is incident.

[0112] The scanning unit 15 includes, for instance, a scanning mirror for scanning the part 18 of the field-of-view 22 with the line of light 5 emitted by the light source 14 along a scanning direction perpendicular to the line of light 5.

[0113] The control 13 obtains the generated histogram data and may output the histogram data via a data bus to an external data processing unit 23.

[0114] An embodiment of a data acquisition of the LiDAR sensor 21 of the LiDAR system 10 is discussed in the following under reference of Fig. 7 in which Fig. 7A shows the embodiment at a first scanning position and Fig. 7B at a second scanning position.

[0115] As mentioned above, the LiDAR sensor 21 includes a plurality of light detection pixels 60, which is arranged in rows (R-l to R-10) and columns.

[0116] The part 62 of the plurality of light detection pixels 60 includes such light detection pixels 60 on which directly reflected illumination light 61 can be incident in accordance with the part 18 of the field-of-view 22 of the imaging device 12 which is illuminated by the illumination device 11 with the line of light 5.

[0117] On the other light detection pixels 60 - not included in the part 62 of the plurality of light detection pixels 60 - no directly reflected illumination light is incident, but indirectly reflected illumination light caused by, for instance, scattering or crosstalk of the directly reflected illumination light.

[0118] In each row (R-l to R-10) of the LiDAR sensor 21, a direct micropixel 70 is configured to acquire a direct signal contribution.

[0119] Moreover, a first global micropixel 72a and a second global micropixel 72b are configured in parts of the LiDAR sensor 21 on which no directly reflected illumination light is incident to acquire a MPI signal contribution.

[0120] According to the first scanning position, as depicted in Fig. 7A, a first subset of light detection pixels 71 of the direct micropixel 70 is enabled in each row (R-l to R-10) and, thus, only light detection events generated by the first subset of light detection pixels 71 contribute to the histogram of the direct micropixel 70.

[0121] At the first scanning position, the first global micropixel 72a is enabled to acquire the MPI signal contribution.

[0122] According to the second scanning position, as depicted in Fig. 7B, a second subset of light detection pixels 71 of the direct micropixel 70 is enabled in each row (R-l to R-10) and, thus, only light detection events generated by the second subset of light detection pixels 71 contribute to the histogram of the direct micropixel 70.

[0123] At the second scanning position, the second global micropixel 72b is enabled to acquire the MPI signal contribution.

[0124] The direct signal contributions of each row (R-l to R-10) and the MPI signal contribution are acquired simultaneously.

[0125] Referring to Fig. 8, which schematically illustrates an embodiment of a light detection pixel 60, the light detection pixel 60 includes a SPAD 50 (e.g., the SPAD 50 of Fig. 4 without limiting the disclosure to this embodiment), a resistor 75 and an AND-gate.

[0126] The anodic voltage is set to GND and the SPAD 50 is reverse-biased via the resistor 75 with a bias voltage Vbias above the breakdown voltage to allow an avalanche current to be generated in the SPAD 50 in response to incident light. Once the avalanche current is generated in response to an incident photon, the cathodic voltage Vc drops until it is below the breakdown voltage. The cathodic voltage Vc is quickly restored to the bias voltage Vbias such that the cathodic voltage Vc includes a short voltage pulse representing a light detection event.

[0127] The light detection pixel 60 obtains an enable control signal EN, e.g., from the control 13.

[0128] The cathodic voltage Vc and the enable control signal EN are connected to the AND-gate such that generated light detection events LDE are only output by the light detection pixel, if the enable control signal EN is HIGH.

[0129] In other embodiments, the enable control signal EN may control a switch (not shown) of the light detection pixel 60 to connect or disconnect the bias voltage Vbias to the resistor 75, thereby activating or deactivating the SPAD 50, respectively. In such embodiments, the electric power consumption may be reduced, since no avalanche current can be generated when the bias voltage Vbias is not applied.

[0130] Referring to Fig. 9, which schematically illustrates an embodiment of the direct macropixel 70 and an embodiment of the global macropixel 72a or 72b, the light detection pixels 60 of the respective macropixel are connected by OR-gates.

[0131] By connecting the outputs of the individual light detection pixels 60 with OR-gates, the direct macropixel 70 and the global macropixel 72a or 72b are configured, since generated LDEs of all connected light detection pixel 60 are included in the output of the respective macropixel. It may be referred to as LDE DMP for the output of the direct macropixel and LDE GMP for the output of the global macropixel 72a or 72b.

[0132] The switching between the global macropixel 72a and 72b may be done by connecting the first global macropixel 72a and the second global macropixel 72b with an OR-gate and by enabling or activating only the light detection pixels 60 of the respective global macropixel 72a or 72b.

[0133] Referring now to Fig. 10, which schematically illustrates in a block diagram an embodiment of the LiDAR sensor 21, the LiDAR sensor 21 includes a counter circuitry 85 for each direct macropixel 70 (DMP in Fig. 9).

[0134] As depicted in Fig. 10, each direct macropixel 70 acquires a direct signal contribution 80 and a MPI signal contribution 81, e.g., a crosstalk contribution.

[0135] The counter circuitry 85 is incremented by the generated LDE DMP of the respective direct macropixel 70 and decremented by the generated LDE GMP of the global macropixel 72, which acquires a MPI signal contribution 82, as depicted in Fig. 10. The same global macropixel 72 is used for each direct macropixel 70.

[0136] The counter circuitry 85 outputs the current value when it is reset by a reset control signal NEW_BIN when a new bin of the histogram 86 starts.

[0137] The histogram 86 is updated with the current value and a MPI signal contribution 83 is suppressed in the final histogram, as depicted in Fig. 9.

[0138] After the last bin, the histogram 86 is output from the LiDAR sensor 21.

[0139] All processing is done directly on-chip such that the effect of the MPI signal is mitigated directly on-chip, which is fast and reduces system complexity.

[0140] Fig. 11 schematically illustrates in a block diagram an embodiment of a spot LiDAR system 110, which is discussed in the following.

[0141] The difference between the spot LiDAR system 110 and the line scanning LiDAR system 10 of Fig. 6 is the illumination device 90 and the LiDAR sensor 92.

[0142] The illumination device 90 includes a driver unit 93 and an array of VCSELs 94, wherein the driver unit 93 drives the array of VCSELs 94 to emit a plurality of light spots within the field-of- illumination 16 towards the scene 17 such that high-intensity areas 95 and low-intensity areas 96 are present in the part 18 of the field-of-view 22.

[0143] The plurality of light spots may further be shifted in at least one predetermined direction to implement a scanning to improve the spatial resolution.

[0144] The LiDAR sensor 92 is configured differently to account for the reflected light spot pattern, as depicted in Fig. 12, which schematically illustrates in a block diagram an embodiment of a data acquisition of the LiDAR sensor 92 of the spot LiDAR system 110, which is discussed in the following.

[0145] A direct macropixel 101 on which directly reflected illumination light 100 is incident is configured in the LiDAR sensor 92 to acquire a direct signal contribution.

[0146] The macropixels 101 are - for the sake of illustration only without limiting the disclosure in this regard - 3x2 -blocks of adjacent light detection pixels 60 instead of a line of adjacent light detection pixels 60 as shown in Fig. 7, wherein enabled light detection pixels 102 may be enabled according to a scanning position of the plurality of light spots.

[0147] Moreover, the LiDAR sensor 92 has configured three global macropixels 103a, 103b and 103c to acquire MPI signal contributions. The global macropixel 103a is used for the direct macropixels in rows R-l to R4, the global macropixel 103b is used for the direct macropixel in rows R-5 to R-6 and the global macropixel 103c is used for the direct macropixels in rows R8 to R-10.

[0148] Fig. 13 schematically illustrates in a flow diagram a method 120 for a LiDAR system, which is discussed in the following.

[0149] The method 120 may be performed by the LiDAR system as described herein.

[0150] At 121, a scene is illuminated with modulated light, as discussed herein.

[0151] At 122, reflected illumination light is imaged on a LiDAR sensor, as discussed herein.

[0152] At 123, a direct signal contribution is acquired to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident, as discussed herein.

[0153] At 124, a multipath interference signal contribution is acquired, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident, as discussed herein.

[0154] At 125, a counter circuitry is incremented by generated light detection events of the direct macropixel and decrementing the counter circuitry by generated light detection events of the global macropixel, as discussed herein.

[0155] At 126, a histogram is updated based on an output of the counter circuitry, as discussed herein.

[0156] At 127, the counter circuitry is resetted for each new bin of the histogram, as discussed herein.

[0157] Returning to the general explanations, summarizing some aspects of some embodiments:

[0158] As discussed herein, the effect of the MPI signal is mitigated by a fast compensation of it on- chip which may allow a framerate to be increased and a system complexity to be reduced.

[0159] A simple and effective on-chip implementation based on counter circuitries is utilized.

[0160] The architecture may suppress MPI signal peaks.

[0161] The on-chip processing counts the light detection events from each direct macropixel during bin time and subtracts the number of light detection events of a global macropixel, which are nonilluminated light detection pixels acquiring only a MPI signal contribution, for suppressing a MPI signal contribution acquired additionally with the direct macropixel.

[0162] The MPI signal contribution may be mitigated without significant impact on system performance and without additional processing or multiple data acquisitions. The LiDAR sensor configuration may be adapted to different types of illumination including flooded or spotted illumination or line illumination or an arbitrary illumination pattern as far as it does not cover the entire field-of-view of the imaging device in a single acquisition.

[0163] It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding.

[0164] It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and / or processors may be used without detracting from the embodiments.

[0165] Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more data processors and / or digital signal processors. The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and / or processors.

[0166] Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the technique.

[0167] Note that the present technology can also be configured as described below.

[0168] (1) A LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident. (2) The LiDAR sensor of (1), wherein the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

[0169] (3) The LiDAR sensor of (1) or (2), including a counter circuitry configured to be incremented by generated light detection events of the direct macropixel and to be decremented by generated light detection events of the global macropixel.

[0170] (4) The LiDAR sensor of any one of (1) to (3), including a plurality of counter circuitries, each configured to be incremented by generated light detection events of an associated direct macropixel and to be decremented by generated light detection events of the global macropixel, wherein the LiDAR sensor is further configured to simultaneously acquire a plurality of direct signal contributions, each with a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitries.

[0171] (5) The LiDAR sensor of (3) or (4), further configured to update a histogram based on an output of the counter circuitry.

[0172] (6) The LiDAR sensor of (5), wherein the counter circuitry is further configured to be resetted for each new bin of the histogram.

[0173] (7) The LiDAR sensor of any one of (1) to (6), including a plurality of light detection pixels, each light detection pixel including a Single-Photon Avalanche Diode.

[0174] (8) The LiDAR sensor of (7), wherein the direct macropixel and the global macropixel, respectively, corresponds to a plurality of adjacent light detection pixels of the plurality of light detection pixels.

[0175] (9) The LiDAR sensor of any one of (1), wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, in particular, wherein the multipath interference signal contribution further includes a contribution from intra-sensor crosstalk.

[0176] (10) A LiDAR system, including: an illumination device configured to illuminate a scene with modulated light; an imaging unit configured to image reflected illumination light on a LiDAR sensor; and the LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident, acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0177] (11) A method for a LiDAR sensor, including: acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

[0178] (12) The method of (11), wherein the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

[0179] (13) The method of (11) or (12), further including incrementing a counter circuitry by generated light detection events of the direct macropixel and decrementing the counter by generated light detection events of the global macropixel.

[0180] (14) The method of any one of (11) to (13), further including incrementing each of a plurality of counter circuitry by generated light detection events of an associated direct macropixel and decrementing the respective counter by generated light detection events of the global macropixel, and simultaneously acquiring a plurality of direct signal contributions, each with a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitries.

[0181] (15) The method of (13) or (14), further including updating a histogram based on an output of the counter circuitry.

[0182] (16) The method of (15), further including resetting the counter circuitry for each new bin of the histogram.

[0183] (17) The method of any one of (11) to (16), wherein the direct macropixel and the global macropixel, respectively, corresponds to a plurality of adjacent light detection pixels of a plurality of light detection pixels. (18) The method of (17), wherein each of the plurality of light detection pixels includes a Single-Photon Avalanche Diode.

[0184] (19) The method of any one of 11, wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, in particular, wherein the multipath interference signal contribution further includes a contribution from intra-sensor crosstalk.

[0185] (20) A method for a LiDAR system, including: illuminating a scene with modulated light; imaging reflected illumination light on a LiDAR sensor; acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

Claims

CLAIMS1. A LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

2. The LiDAR sensor of claim 1, wherein the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

3. The LiDAR sensor of claim 1, comprising a counter circuitry configured to be incremented by generated light detection events of the direct macropixel and to be decremented by generated light detection events of the global macropixel.

4. The LiDAR sensor of claim 1, comprising a plurality of counter circuitries, each configured to be incremented by generated light detection events of an associated direct macropixel and to be decremented by generated light detection events of the global macropixel, wherein the LiDAR sensor is further configured to simultaneously acquire a plurality of direct signal contributions, each with a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitries.

5. The LiDAR sensor of claim 3, further configured to update a histogram based on an output of the counter circuitry.

6. The LiDAR sensor of claim 5, wherein the counter circuitry is further configured to be resetted for each new bin of the histogram.

7. The LiDAR sensor of claim 1, comprising a plurality of light detection pixels, each light detection pixel including a Single-Photon Avalanche Diode.

8. The LiDAR sensor of claim 7, wherein the direct macropixel and the global macropixel, respectively, corresponds to a plurality of adjacent light detection pixels of the plurality of light detection pixels.

9. The LiDAR sensor of claim 1, wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, in particular, wherein themultipath interference signal contribution further includes a contribution from intra-sensor crosstalk.

10. A LiDAR system, comprising: an illumination device configured to illuminate a scene with modulated light; an imaging unit configured to image reflected illumination light on a LiDAR sensor; and the LiDAR sensor configured to: acquire a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident, acquire a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

11. A method for a LiDAR sensor, comprising: acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

12. The method of claim 11, wherein the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

13. The method of claim 11, further comprising incrementing a counter circuitry by generated light detection events of the direct macropixel and decrementing the counter by generated light detection events of the global macropixel.

14. The method of claim 11, further comprising incrementing each of a plurality of counter circuiries by generated light detection events of an associated direct macropixel and decrementing the respective counter by generated light detection events of the global macropixel, and simultaneously acquiring a plurality of direct signal contributions, each with a differentdirect macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitries.

15. The method of claim 13, further comprising updating a histogram based on an output of the counter circuitry.

16. The method of claim 15, further comprising resetting the counter circuitry for each new bin of the histogram.

17. The method of claim 11, wherein the direct macropixel and the global macropixel, respectively, corresponds to a plurality of adjacent light detection pixels of a plurality of light detection pixels.

18. The method of claim 17, wherein each of the plurality of light detection pixels includes a Single-Photon Avalanche Diode.

19. The method of claim 11, wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, in particular, wherein the multipath interference signal contribution further includes a contribution from intra-sensor crosstalk.

20. A method for a LiDAR system, comprising: illuminating a scene with modulated light; imaging reflected illumination light on a LiDAR sensor; acquiring a direct signal contribution, in a region of interest inside the actively illuminated part of the LiDAR sensor, to obtain distance information, wherein the direct signal contribution is acquired with a direct macropixel on which directly reflected illumination light is incident; and acquiring a multipath interference signal contribution, in a region of interest outside the actively illuminated part of the LiDAR sensor, wherein the multipath interference signal contribution is acquired with a global macropixel on which indirectly reflected illumination light is incident.

Images