LiDAR sensor, LiDAR system and method

Abstract

A LiDAR sensor configured to obtain direct signal contributions in a region of interest inside an actively illuminated portion of the LiDAR sensor to obtain distance information, wherein the direct signal contributions are obtained with direct macro-pixels that are hit by directly reflected illumination light, and to obtain multi-path interference signal contributions in a region of interest outside the actively illuminated portion of the LiDAR sensor, wherein the multi-path interference signal contributions are obtained with global macro-pixels that are hit by indirectly reflected illumination light.

Description

Technical Field

[0001] This disclosure generally relates to LiDAR sensors, LiDAR systems, methods for LiDAR sensors, and methods for LiDAR systems. Background Technology

[0002] Typically, direct time-of-flight ("dToF") systems are known, which can also be referred to as LiDAR ("light detection and ranging"). These dToF systems are commonly used to determine the distance to objects in a scene.

[0003] However, for example, when a LiDAR system illuminates an area of ​​the scene larger than the field of view of a single sensor element, the light-detecting pixels may receive multipath interference (“MPI”) signals from other areas of the scene. This could be caused, for example, by optical crosstalk in the receiver-side optical system or by multiple scattering events of the illuminating light at multiple objects in the scene. MPI signals can reduce sensing accuracy, and therefore it is desirable to mitigate the impact of MPI signals on the determined distance.

[0004] A method for mitigating the effects of MPI signals by using a first and a second bin of photodetector pixels is known from international patent application publication WO 2023 / 041465 A1. Some embodiments of this disclosure can avoid frame rate limitations, thereby allowing operation at higher frame rates and / or potentially improving power efficiency in certain situations, and can avoid and / or limit the need for post-processing of the acquired signals and histograms.

[0005] Some embodiments of this disclosure can mitigate the impact of MPI signals by performing fast on-chip compensation for the effects of MPI signals, thereby allowing for increased frame rates and reduced system complexity.

[0006] Although technologies exist for LiDAR systems, improvements to existing technologies are generally expected. Summary of the Invention

[0007] According to a first aspect, this disclosure provides a LiDAR sensor configured to: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0008] According to a second aspect, this disclosure provides a LiDAR system comprising: The illumination device is configured to illuminate the scene using modulated light; The imaging unit is configured to image the reflected illumination light onto a LiDAR sensor; and The LiDAR sensor is configured as follows: In the region of interest within the active illumination section of the LiDAR sensor, the direct signal contribution is acquired to obtain distance information. This direct signal contribution is obtained using the direct macropixels to which the directly reflected illumination light is incident. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0009] According to a third aspect, this disclosure provides a method for a LiDAR sensor, the method comprising: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0010] According to the fourth aspect, this disclosure provides a method for a LiDAR system, the method comprising: Illuminate the scene using modulated light; The reflected illumination light is imaged onto the LiDAR sensor; In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0011] Other aspects are set forth in the dependent claims, the drawings and the following description. Attached Figure Description

[0012] The accompanying drawings illustrate the implementation methods by way of example, in which: Figure 1The schematic diagram illustrates the operating principle of the LiDAR system; Figure 2 An example of the occurrence of multipath interference signals is illustrated schematically; Figure 3 An embodiment of the imaging device is illustrated schematically in a block diagram, wherein the indirectly reflected illumination light is caused by crosstalk of the directly reflected illumination light. Figure 4 A block diagram schematically illustrates an implementation of a LiDAR sensor based on a single-photon avalanche diode, wherein crosstalk within the sensor contributes to the multipath interference signal. Figure 5 A block diagram schematically illustrates an implementation of a LiDAR sensor; Figure 6 A block diagram schematically illustrates an implementation of a line-scan LiDAR system; Figure 7 exist Figure 7 Figure A schematically illustrates, in a block diagram, an implementation method for data acquisition by the LiDAR sensor at the first scan position in a line-scan LiDAR system, and... Figure 7 B schematically illustrates, in block diagram, an implementation method for data acquisition by the LiDAR sensor at the second scan position in a line-scan LiDAR system; Figure 8 An implementation of the light detection pixel is schematically illustrated; Figure 9 The implementation methods of direct macropixels and global macropixels are illustrated schematically. Figure 10 A block diagram schematically illustrates an implementation of a LiDAR sensor; Figure 11 A block diagram schematically illustrates the implementation of the spot LiDAR system; Figure 12 A block diagram schematically illustrates an implementation method for data acquisition by a LiDAR sensor in a spot LiDAR system; and Figure 13 A flowchart schematically illustrates the method used for the LiDAR system. Detailed Implementation

[0013] Provide a reference Figure 6 Before a detailed description of the implementation methods, a general explanation will be given.

[0014] As mentioned at the beginning, direct time-of-flight ("dToF") systems, also known as LiDAR ("light detection and ranging"), are generally known. These dToF systems are typically used to determine the distance to objects in a scene.

[0015] To enhance the overall understanding of this disclosure, reference is made below. Figure 1 The operating principle of LiDAR system 1 is discussed. The figure schematically illustrates the operating principle and can also be applied to other embodiments of this disclosure.

[0016] The LiDAR system 1 includes an irradiation device 2 and an imaging device 3 controlled by a control unit (not shown), wherein the imaging device 3 includes a LiDAR sensor (not shown).

[0017] Example LiDAR system 1 is based on dToF technology and uses a single-photon avalanche diode (“SPAD”) array as a LiDAR sensor for its operation of acquiring depth data.

[0018] like Figure 1 The described dToF technology is based on the process of illuminating scene 4 by illumination device 2 and detecting the reflected illumination light returning from scene 4 by imaging device 3.

[0019] For example, the process begins with the emission of short light pulses toward scene 4. As these light pulses interact with objects in scene 4, a portion of the photons are reflected toward imaging device 3. The reflected illumination is detected by a SPAD with a probability of forming an avalanche current for each received photon.

[0020] The LiDAR sensor then records the arrival time of each light detection event and groups them into discrete time intervals (bins) to create a histogram, thereby generating ToF data (histogram data).

[0021] To improve the signal-to-noise ("SNR") ratio, the process can be repeated several times, and thus the final histogram can be the sum of the histograms of each emitted light pulse.

[0022] Subsequently, the ToF data (histogram data) is processed to detect peaks in the histogram that indicate the arrival time of the reflected pulses, and thus the distance (depth data) of objects in the scene is detected.

[0023] Each light-detecting pixel in a LiDAR sensor can include one or more SPADs, and a histogram is constructed for each pixel (by summing the light-detecting events of each SPAD in the light-detecting pixel) to obtain a 3D point cloud of the target field of view of the LiDAR system.

[0024] However, returning to a general explanation, as further mentioned at the beginning, when a LiDAR system illuminates an area of ​​the scene larger than the field of view of a single sensor element, the light-detecting pixels may receive multipath interference (“MPI”) signals from other areas of the scene. This could be caused, for example, by optical crosstalk in the receiver-side optical system or by multiple scattering of the illuminating light from objects in the scene.

[0025] Because LiDAR systems typically acquire Time-of-Flight (ToF) data for the entire field of view or a portion thereof simultaneously, in non-ideal systems, reflected light from different points in the scene may interfere with each other. This can cause the histogram to show a direct peak belonging to the target region of the scene and an indirect peak belonging to other parts of the scene; however, in an ideal system, there should be no indirect peaks in the histogram.

[0026] The MPI signal may reduce sensing accuracy, and therefore it is desirable to mitigate the impact of the MPI signal on the determined distance.

[0027] To further enhance the overall understanding of this disclosure, Figure 2 The diagram illustrates an example of the occurrence of an MPI signal, which will be discussed below.

[0028] A line-scan LiDAR system (not shown) illuminates a scene with a line of light 5, in which a highly reflective object 6 (here, a traffic sign) exists. The scanning direction of the line-scan LiDAR system is perpendicular to the line of light 5. It should be understood that object 6 can be another object without intentional reflective properties.

[0029] In a line-scan LiDAR system, the first light-detecting pixel of the LiDAR sensor acquires the direct signal contribution from the illumination light directly reflected at object point 7.

[0030] The scene also includes a dark background 8a above object 6 and a dark background 8b below it, from which no light is reflected directly. Of course, if any objects exist within the field of view representing 8a and 8b, these objects may have some degree of reflection.

[0031] However, the illumination light directly reflected from object point 7 causes optical crosstalk within the imaging unit of the LiDAR system. Optical (lens) crosstalk can be generated by diffraction effects, geometric aberrations, and stray light in the imaging lens. Stray light can be caused by lens surface roughness and multiple reflections within the lens element. Diffraction imposes a physical constraint on the lens's point spread function ("PSF"), which is defined by the lens's f-number, and geometric aberrations, caused by defects in the lens design, will also lead to a finite PSF.

[0032] Due to optical crosstalk, the indirectly reflected illumination light is incident on the second photodetector pixel near the first photodetector pixel. Therefore, an MPI signal contribution exists at the second photodetector pixel.

[0033] The imaging unit of the LiDAR system images the illumination light directly reflected from scene region 9 onto the second light detection pixel. Because of the dark background in scene region 9, the second light detection pixel does not acquire a direct signal contribution; however, as discussed above, it acquires an MPI signal contribution.

[0034] Therefore, the histogram of the second light-detecting pixel indicates an unconfirmed object point in scene region 9, which does not actually exist. Consequently, object 6 appears larger in the vertical direction than it actually is, which could lead to inaccurate decisions by advanced driver assistance systems (ADAS) that use depth data from the LiDAR system.

[0035] In addition, indirectly reflected illumination light may also strike the first light detection pixel, caused, for example, by crosstalk or scattering of illumination light directly reflected from an object point 6 located above or below object point 7 and also illuminated by linear light 5. Furthermore, this indirectly reflected illumination may also strike the second light detection pixel.

[0036] To further enhance the overall understanding of this disclosure, reference is made below. Figure 3 An embodiment of the imaging device 12, which is discussed in this paper, is illustrated schematically in block diagram form, and the crosstalk between the indirectly reflected illumination light and the directly reflected illumination light is caused by the indirect reflection illumination light.

[0037] The imaging device 12 includes an imaging unit 20 and a LiDAR sensor 21, wherein the LiDAR sensor 21 is arranged in the focal plane of the imaging unit, which is for illustrative purposes of optical crosstalk and not to limit the present disclosure. The distance between the imaging unit 20 and the LiDAR sensor 21 may be adjustable, and the optical properties of the imaging unit 20 may be adjustable, for example, by lens movement or by using a liquid lens.

[0038] The imaging unit 20 includes a first lens 30, a second lens 31, and a third lens 32 that image light rays 40a and 40b passing through the opening (not shown) onto the LiDAR sensor 21.

[0039] As the light rays travel through the lens stack, ray 40a and 40b are reflected at the lens surfaces (e.g., at surface (e)). For example, the reflected rays 41a and 41b are reflected again at surface (b), causing them to strike the LiDAR sensor 21 at a different location than rays 40a and 40b.

[0040] Therefore, the illumination light directly reflected by the first light detection pixel of the LiDAR sensor 21 is indirectly reflected by optical crosstalk and incident on the second light detection pixel of the LiDAR sensor 21.

[0041] To further enhance the overall understanding of this disclosure, reference is made to... Figure 4 An implementation of a SPAD-based LiDAR sensor 21 in which crosstalk within the sensor contributes to the MPI signal is discussed. This implementation is schematically illustrated in block diagram and can also be applied to other implementations of this disclosure.

[0042] When light is incident on SPAD 50, photogenerated electrons 51 may exist and may diffuse into the depletion region 52 of SPAD 50.

[0043] Once the photogenerated electron 51 reaches the depletion region 52, an avalanche process may occur that can generate additional electrons 53 and 55, as is generally known.

[0044] Typically, these electrons 53 and 55 will be driven toward the cathode, causing a voltage drop at the cathode and causing the SPAD 50 to output an avalanche signal until the cathode voltage drops below the breakdown voltage and the bias voltage recovers at the cathode.

[0045] However, electrons 53 and 55 may cause crosstalk within the sensor because photons are emitted during the avalanche process due to electroluminescence, and some of these photons may reach neighboring SPADs via direct or indirect paths.

[0046] Indirect paths typically involve at least one reflection at the bottom layer of SPAD 50, where a resonance peak can be found for total internal reflection conditions, which depend on the refractive index differences between layers and the incident angle of the generated photons.

[0047] The direct path may cause photogeneration of electron 54 in a nearby SPAD, which may diffuse into the depletion region and trigger an avalanche process.

[0048] Furthermore, electrons 53 and 55 may directly diffuse into the adjacent SPAD, where they may diffuse into the depletion zone and trigger an avalanche process.

[0049] Returning to the general interpretation, it has been recognized that the effects of the MPI signal should be mitigated, as will be discussed below.

[0050] As mentioned at the beginning of International Patent Application Publication WO 2023 / 041465 A1, a method for mitigating the effects of MPI signals by using a first and second bin of light-detecting pixels is known. Some embodiments of this disclosure can avoid frame rate limitations, thereby allowing operation at higher frame rates and / or potentially improving power efficiency in certain situations, and can avoid and / or limit the need for post-processing of the acquired signals and histograms.

[0051] Therefore, the goal is to mitigate the impact of the MPI signal by performing fast on-chip compensation, thereby allowing for increased frame rates and reduced system complexity.

[0052] Therefore, some implementations relate to a LiDAR system, wherein the LiDAR system includes: The illumination device is configured to illuminate the scene using modulated light; The imaging unit is configured to image the reflected illumination light onto a LiDAR sensor; and The LiDAR sensor is configured as follows: In the region of interest within the active illumination section of the LiDAR sensor, the direct signal contribution is acquired to obtain distance information. This direct signal contribution is obtained using the direct macropixels to which the directly reflected illumination light is incident. In the region of interest outside the active illumination portion of the LiDAR sensor, the MPI signal contribution is acquired, where the MPI signal contribution is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0053] Therefore, some implementations relate to a LiDAR sensor, wherein the LiDAR sensor is configured as follows: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0054] The irradiation device may include an irradiator, which may include one or more light sources, such as one or more light-emitting diodes (LEDs), one or more laser diodes, one or more vertical cavity surface-emitting lasers (VCSELs), etc.

[0055] An irradiator may include one or more drivers to drive each light source independently according to a corresponding control signal, or to drive all light sources to emit modulated light according to a control signal.

[0056] Irradiation devices may include optical components such as lenses (such as glass, plastic, or liquid lenses), mirrors, filters, etc.

[0057] The irradiation device may include mechanical parts that move optical components, such as piezoelectric actuators.

[0058] Modulated light can be modulated in time and space, such that the modulated light includes, for example, a sequence of emitted light pulses as a line of light, as multiple light spots, or as a spatial Gaussian distribution (also known as floodlighting).

[0059] Imaging units may include optical components such as lenses (such as glass, plastic, or liquid lenses), mirrors, filters, etc.

[0060] The imaging unit may include mechanical parts that move optical components, such as piezoelectric actuators.

[0061] A LiDAR system may also include controls configured to control the overall operation of the LiDAR system, including, for example, light emission timing, data acquisition, and synchronization between the illumination device and the LiDAR sensor, as is commonly known.

[0062] A LiDAR sensor includes multiple light-detecting pixels, which can be arranged in rows or in an array of rows and columns.

[0063] In some implementations, each light-detecting pixel includes a SPAD. Each SPAD is configured to generate a light-detecting event in response to incident light, as is generally known.

[0064] Each of the multiple light-detecting pixels in a LiDAR sensor receives illumination light directly reflected from a different predetermined area of ​​the scene, which is illuminated and located within the field of view of the corresponding light-detecting pixel (assuming a reflective object exists in the predetermined area of ​​the scene).

[0065] Therefore, the LiDAR sensor is configured to acquire a direct signal contribution in a region of interest within the active illumination portion of the LiDAR sensor, to obtain distance information, for example, based on the arrival time of the directly reflected illumination light relative to the emission start timing of the modulated light.

[0066] LiDAR sensors utilize direct macropixels to capture direct signal contributions.

[0067] In some implementations, a direct macropixel corresponds to multiple adjacent light-detecting pixels among a plurality of light-detecting pixels. The light-detecting events generated for each light-detecting pixel are binned.

[0068] The region of interest (ROI) within the active illumination portion of a LiDAR sensor can be any part of the LiDAR sensor where direct reflection of the illumination light occurs. The ROI can, for example, depend on the scanning position of the LiDAR system.

[0069] However, as discussed above, direct macropixels may also acquire MPI signal contributions, which could reduce sensing accuracy.

[0070] Therefore, LiDAR sensors utilize different macropixels to obtain MPI signal contributions. Specifically, they utilize global macropixels that have never received directly reflected illumination light because their field of view has never been directly illuminated by the LiDAR system.

[0071] Therefore, the MPI signal contribution obtained using global macropixels is used to estimate the MPI signal contribution obtained using direct macropixels, in order to mitigate the impact of MPI on sensing accuracy.

[0072] In some implementations, a global macropixel corresponds to multiple adjacent light-detecting pixels among a plurality of light-detecting pixels. The light-detecting events generated for each light-detecting pixel are binned.

[0073] In some implementations, the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light. Specifically, the contribution of the multipath interference signal also includes contributions from crosstalk within the sensor, as discussed above.

[0074] To further enhance the overall understanding of this disclosure, reference is made to... Figure 5 An implementation of the LiDAR sensor 200 is discussed. The figure schematically illustrates this implementation as a block diagram, and it can also be applied to other implementations of this disclosure.

[0075] The LiDAR sensor 200 has an actively illuminated portion 201. In other words, portion 201 receives directly reflected illumination light, while the rest of the LiDAR sensor (the portion of the LiDAR sensor located outside the actively illuminated portion 201) receives only indirectly reflected illumination light.

[0076] Therefore, the global macropixel 202 is configured outside the actively illuminated portion 201, because this portion allows the acquisition of MPI signal contributions.

[0077] Returning to the general interpretation, in order to allow for increased frame rates, in some implementations, the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

[0078] In some implementations, the LiDAR sensor includes a counter circuit system configured to increment due to light detection events of generated direct macropixels and decrement due to light detection events of generated global macropixels.

[0079] This allows for direct, on-chip, rapid compensation of MPI signal contributions, thereby enabling increased frame rates and reduced system complexity.

[0080] Since the MPI signal contribution of global macropixels is directly subtracted on-chip, the MPI signal contribution obtained using direct macropixels is expected to decrease.

[0081] Furthermore, when the global macropixel acquisition does not utilize the MPI signal contribution that does not overlap with the acquired direct signal contribution in time, the counter circuit system only decrements the count to zero for the corresponding bin, so that the resulting histogram is not negatively affected.

[0082] Furthermore, even if the MPI signal contribution obtained using global macropixels is not obtained using direct macropixels and overlaps with the obtained direct signal contribution in time, the impact on peak detection is expected to be small, because the amount of light from directly reflected illumination is usually much higher than that from indirectly reflected illumination.

[0083] Therefore, the peaks in the histogram corresponding to the direct signal contribution will only decrease slightly, and thus peak detection will not be significantly affected.

[0084] In some implementations, the LiDAR sensor is also configured to update the histogram based on the output of a counter circuit system.

[0085] Therefore, in some cases, signal post-processing may be unnecessary or minimal, thereby reducing system complexity.

[0086] In some implementations, the counter circuitry is also configured to reset for each new bin in the histogram.

[0087] Therefore, a time-to-digital converter (“TDC”) may not be needed, thereby reducing system complexity.

[0088] Similarly, multiple different direct macropixels can be used to obtain multiple different direct signal contributions.

[0089] Therefore, in some embodiments, the LiDAR sensor includes a plurality of counter circuitry systems, each configured to increment due to light detection events of associated direct macropixels and decrement due to light detection events of generated global macropixels. The LiDAR sensor is also configured to simultaneously acquire a plurality of direct signal contributions, each acquired using a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuitry systems.

[0090] Some implementations relate to a method for a LiDAR sensor, the method comprising: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0091] This method can be performed by a LiDAR sensor as described in this paper.

[0092] Some implementations relate to a method for a LiDAR system, the method comprising: Illuminate the scene using modulated light; The reflected illumination light is imaged onto the LiDAR sensor; In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0093] This method can be performed by a LiDAR system as described in this paper.

[0094] Any LiDAR sensor discussed in this article may also be referred to as a light receiver or a light detection sensor.

[0095] Back Figure 6 This figure schematically illustrates an implementation of the line-scan LiDAR system 10 using a block diagram. See below for reference. Figures 6 to 10 This needs to be discussed.

[0096] The line-scan LiDAR system 10 includes an illumination device 11 and an imaging device 12 (e.g., Figure 3 The imaging device 12 (without limiting the present disclosure to this embodiment) and control 13.

[0097] The control unit 13 essentially controls the overall operation of the line scan LiDAR system 10, such as light emission timing, data acquisition, and synchronization between the illumination device 11 and the imaging device 12, as is generally known.

[0098] The irradiation device 11 includes a light source 14 and a scanning unit 15.

[0099] Imaging device 12 includes imaging unit 20 and LiDAR sensor 21, wherein LiDAR sensor 21 includes a plurality of light detection pixels, each light detection pixel including a SPAD (e.g., Figure 4 (This disclosure is not limited to this embodiment of SPAD).

[0100] The light source 14 includes a laser diode configured to emit a linear light 5 (e.g., a 1D illumination pattern) toward a scene 17 where an object 19 is present within an illumination field 16.

[0101] Any light originating from scene 17 is collected by imaging device 12 when it is within the field of view 22 of imaging device 12.

[0102] The illumination device 11 illuminates only a portion 18 of the field of view 22 of the imaging device 12, such that the LiDAR sensor 21 includes a light detection pixel area to which the directly reflected illumination light is not incident.

[0103] For example, the scanning unit 15 includes a scanning mirror for scanning a portion 18 in the field of view 22 along a scanning direction perpendicular to the linear light 5 emitted by the light source 14.

[0104] The control unit 13 obtains the generated histogram data and can output the histogram data to the external data processing unit 23 via the data bus.

[0105] The following is for reference. Figure 7 This paper discusses the implementation method of data acquisition by the LiDAR sensor 21 in the LiDAR system 10. In this figure, Figure 7 A illustrates an implementation at the first scan position and Figure 7 B illustrates the implementation at the second scan position.

[0106] As mentioned above, the LiDAR sensor 21 includes a plurality of light-detecting pixels 60 arranged in rows (R-1 to R-10) and columns.

[0107] A portion 62 of the plurality of light detection pixels 60 includes light detection pixels 60 such that directly reflected illumination light 61 may be incident on these light detection pixels according to the portion 18 of the illuminated device 11 in the field of view 22 of the imaging device 12 illuminated by the linear light 5.

[0108] The directly reflected illumination light does not strike the other light detection pixels 60, but the indirectly reflected illumination light, caused by scattering or crosstalk of, for example, the directly reflected illumination light, strikes these other light detection pixels, which are not included in the portion 62 of the plurality of light detection pixels 60.

[0109] In each row (R-1 to R-10) of the LiDAR sensor 21, direct micropixels 70 are configured to acquire direct signal contributions.

[0110] Furthermore, the first global micropixel 72a and the second global micropixel 72b are configured in the portion of the LiDAR sensor 21 where the illumination light directly reflected is not incident, in order to obtain the MPI signal contribution.

[0111] Based on the first scan position, such as Figure 7 As depicted in A, the first light-detecting pixel subset 71 of the direct micropixel 70 is enabled in each row (R-1 to R-10), and therefore only the light-detecting events generated by the first light-detecting pixel subset 71 contribute to the histogram of the direct micropixel 70.

[0112] At the first scan position, the first global micropixel 72a is enabled to obtain the MPI signal contribution.

[0113] According to the second scan position, such as Figure 7 As described in B, the second light-detection pixel subset 71 of the direct micropixel 70 is enabled in each row (R-1 to R-10), and therefore only the light-detection events generated by the second light-detection pixel subset 71 contribute to the histogram of the direct micropixel 70.

[0114] At the second scan position, the second global micropixel 72b is enabled to obtain the MPI signal contribution.

[0115] The direct signal contribution and MPI signal contribution for each row (R-1 to R-10) are acquired simultaneously.

[0116] refer to Figure 8 The figure schematically illustrates an embodiment of the light detection pixel 60, which includes a SPAD 50 (e.g., Figure 4 SPAD 50, without limiting this disclosure to this embodiment), resistor 75 and AND gate.

[0117] The anode voltage is set to GND, and SPAD 50 is biased via resistor 75 at a voltage V higher than the breakdown voltage. 偏置 It is reverse biased to allow avalanche current to be generated in response to incident light in the SPAD 50.

[0118] Once an avalanche current is generated in response to the incident photon, the cathode voltage Vc drops until it falls below the breakdown voltage. The cathode voltage Vc quickly recovers to the bias voltage V. 偏置 This causes the cathode voltage Vc to include a short voltage pulse representing a photodetection event.

[0119] The light detection pixel 60 obtains the enable control signal EN from the control element 13, for example.

[0120] The cathode voltage Vc and the enable control signal EN are connected to an AND gate, so that the photodetector pixel outputs the generated photodetector event LDE only when the enable control signal EN is high (HIGH).

[0121] In other embodiments, the enable control signal EN can control the switching of the light detection pixel 60 (not shown) to adjust the bias voltage V. 偏置 Connecting to or disconnecting resistor 75 activates or deactivates SPAD 50, respectively. In such an implementation, since no bias voltage V is applied... 偏置 It does not generate avalanche current, thus reducing power consumption.

[0122] refer to Figure 9 The figure schematically illustrates an implementation of the direct macropixel 70 and an implementation of the global macropixel 72a or 72b, wherein the light detection pixels 60 of the corresponding macropixels are connected via OR gates.

[0123] By connecting the outputs of each light-detecting pixel 60 with an OR gate, direct macropixels 70 and global macropixels 72a or 72b are configured, since the LDEs of all connected light-detecting pixels 60 are included in the outputs of the corresponding macropixels. The output of the direct macropixel can be referred to as LDE_DMP, and the output of the global macropixel 72a or 72b can be referred to as LDE_GMP.

[0124] Switching between global macro pixels 72a and 72b can be accomplished by connecting the first global macro pixel 72a and the second global macro pixel 72b with an OR gate and by enabling or activating only the light detection pixel 60 of the corresponding global macro pixel 72a or 72b.

[0125] Now for reference Figure 10 The figure schematically illustrates an implementation of a LiDAR sensor 21, which includes components for each direct macropixel 70 (…). Figure 9 The counter circuit system 85 of the DMP (Dual Multiprocessor) in the system.

[0126] like Figure 10 As depicted, each direct macropixel 70 acquires a direct signal contribution 80 and an MPI signal contribution 81, such as a crosstalk contribution.

[0127] The counter circuit system 85 increments due to the LDE_DMP of the corresponding direct macropixel 70 generated, and decrements due to the LDE_GMP of the global macropixel 72 generated by the acquisition MPI signal contribution 82, such as Figure 10 As depicted, the same global macro pixel 72 is used for each direct macro pixel 70.

[0128] At the start of the new bin in histogram 86, the counter circuit system 85 outputs the current value when it is reset by the reset control signal New_BIN.

[0129] The histogram 86 is updated using the current value, and the MPI signal contribution 83 is suppressed in the final histogram, as shown below. Figure 9 What is depicted.

[0130] After the last box, histogram 86 is output from LiDAR sensor 21.

[0131] All processing is performed directly on-chip, which mitigates the impact of the MPI signal by doing so directly on-chip. This is fast and reduces system complexity.

[0132] Figure 11 A block diagram schematically illustrates an implementation of the spot LiDAR system 110 discussed below.

[0133] The LiDAR system 110 with light spot Figure 6 The difference between the line scan LiDAR systems 10 lies in the illumination device 90 and the LiDAR sensor 92.

[0134] The illumination device 90 includes a driver unit 93 and a VCSEL array 94, wherein the driver unit 93 drives the VCSEL array 94 to emit multiple light spots toward the scene 17 within the illumination field 16, so that a high-intensity region 95 and a low-intensity region 96 exist in a portion 18 of the field of view 22.

[0135] Multiple light spots can be further shifted in at least one predetermined direction to achieve scanning, thereby improving spatial resolution.

[0136] The LiDAR sensor 92 is configured differently to take into account the reflected light spot pattern, such as Figure 12The figure depicts, schematically illustrating in block form, an implementation of the data acquisition method of the LiDAR sensor 92 in the spot LiDAR system 110, which will be discussed below.

[0137] In the LiDAR sensor 92, direct reflected illumination light 100 is incident on direct macropixels 101 to obtain direct signal contribution.

[0138] For illustrative purposes only and not to limit this disclosure, macropixel 101 is a 3×2 block of adjacent light-detecting pixels 60, rather than as shown in the diagram. Figure 7 The row of adjacent light detection pixels 60 shown can be used to enable the light detection pixels 102 based on the scanning position of multiple light spots.

[0139] In addition, the LiDAR sensor 92 is configured with three global macro pixels 103a, 103b and 103c to capture MPI signal contributions.

[0140] Global macro pixel 103a is used for direct macro pixels in rows R-1 to R-4, global macro pixel 103b is used for direct macro pixels in rows R-5 to R-6, and global macro pixel 103c is used for direct macro pixels in rows R-8 to R-10.

[0141] Figure 13 A flowchart schematically illustrates method 120 for a LiDAR system, which will be discussed below.

[0142] Method 120 can be performed by a LiDAR system as described herein.

[0143] At position 121, the scene is illuminated using modulated light, as discussed in this paper.

[0144] At position 122, the reflected illumination light is imaged onto the LiDAR sensor, as discussed in this paper.

[0145] At position 123, the direct signal contribution is obtained to acquire distance information. The direct signal contribution is obtained by using the direct macropixel to which the illumination light is incident, as discussed in this paper.

[0146] At position 124, the contribution of the multipath interference signal is obtained, which is obtained by using the incident global macro-pixel of the indirectly reflected illumination light, as discussed in this paper.

[0147] At 125, the counter circuitry increments due to the light detection events of the generated direct macropixels and decrements due to the light detection events of the generated global macropixels, as discussed herein.

[0148] At position 126, the histogram is updated based on the output of the counter circuit system, as discussed in this paper.

[0149] At position 127, the counter circuitry is reset for each new bin in the histogram, as discussed in this paper.

[0150] Returning to a general explanation, let's summarize some aspects of the implementation methods: As discussed in this paper, mitigating the impact of MPI signals through on-chip fast compensation can allow for increased frame rates and reduced system complexity.

[0151] A simple and effective on-chip implementation based on a counter circuit system is used.

[0152] This architecture can suppress MPI signal peaks.

[0153] On-chip processing counts the light detection events from each direct macropixel during the bin time and subtracts the number of light detection events from the global macropixel to suppress the additional MPI signal contribution acquired by the direct macropixel, which is a non-illuminated light detection pixel that only acquires the MPI signal contribution.

[0154] It can reduce the contribution of MPI signals without significantly affecting system performance and without additional processing or multiple data acquisitions.

[0155] LiDAR sensor configurations can be adapted to different types of illumination, including flood illumination, spot illumination, line illumination, or arbitrary illumination patterns, as long as they do not cover the entire field of view of the imaging device in a single acquisition.

[0156] It should be recognized that the implementation methods describe the method using an exemplary order of method steps. However, the specific order of method steps given is for illustrative purposes only and should not be construed as binding.

[0157] It should be understood that, for clarity, the above description has referenced different functional units, circuit systems, and / or processors in describing implementations. However, it will be apparent that any suitable functional distribution among the different functional units, circuit systems, and / or processors can be used without diminishing the implementation.

[0158] The described embodiments can be implemented in any suitable form, including hardware, software, firmware, or any combination thereof. The described embodiments can optionally be implemented, at least in part, as computer software running on one or more data processors and / or digital signal processors. Elements and components of any embodiment can be implemented physically, functionally, and logically in any suitable manner. In practice, functionality can be implemented in a single unit, in multiple units, or as part of other functional units. Therefore, the disclosed embodiments can be implemented in a single unit or can be physically and functionally distributed among different units, circuit systems, and / or processors.

[0159] Although this disclosure has been described in conjunction with some embodiments, it is not intended to be limited to the specific forms set forth herein. Furthermore, while features may appear to be described in conjunction with specific embodiments, those skilled in the art will recognize that the various features of the described embodiments can be combined in any manner suitable for implementing the present technology.

[0160] It should be noted that this technology can also be configured as described below.

[0161] (1) A LiDAR sensor configured as follows: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0162] (2) According to the LiDAR sensor in (1), the direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

[0163] (3) The LiDAR sensor according to (1) or (2) includes a counter circuit system configured to increment due to light detection events of generated direct macropixels and decrement due to light detection events of generated global macropixels.

[0164] (4) The LiDAR sensor according to any one of (1) to (3) includes a plurality of counter circuit systems, each counter circuit system being configured to increment due to light detection events of generated associated direct macropixels and decrement due to light detection events of generated global macropixels, wherein the LiDAR sensor is further configured to simultaneously acquire a plurality of direct signal contributions, each direct signal contribution being acquired using different direct macropixels, wherein each direct macropixel is associated with one of the plurality of counter circuit systems.

[0165] (5) The LiDAR sensor according to (3) or (4) is also configured to update the histogram based on the output of the counter circuit system.

[0166] (6) According to the LiDAR sensor in (5), the counter circuit system is also configured to reset for each new bin of the histogram.

[0167] (7) A LiDAR sensor according to any one of (1) to (6) includes a plurality of light detection pixels, each light detection pixel including a single-photon avalanche diode.

[0168] (8) According to the LiDAR sensor of (7), the direct macropixel and the global macropixel correspond to multiple adjacent light detection pixels among multiple light detection pixels.

[0169] (9) A LiDAR sensor according to any one of (1), wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, specifically, wherein the contribution of the multipath interference signal also includes the contribution from crosstalk within the sensor.

[0170] (10) A LiDAR system, comprising: The illumination device is configured to illuminate the scene using modulated light; The imaging unit is configured to image the reflected illumination light onto a LiDAR sensor; and The LiDAR sensor is configured as follows: In the region of interest within the active illumination section of the LiDAR sensor, the direct signal contribution is acquired to obtain distance information. This direct signal contribution is obtained using the direct macropixels to which the directly reflected illumination light is incident. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0171] (11) A method for a LiDAR sensor, comprising: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

[0172] (12) According to the method of (11), the direct signal contribution and the multipath interference signal contribution are obtained simultaneously.

[0173] (13) According to the method of (11) or (12), it also includes incrementing the counter circuit system due to the light detection event of the generated direct macropixel and decrementing the counter due to the light detection event of the generated global macropixel.

[0174] (14) The method according to any one of (11) to (13) further includes incrementing each of the plurality of counter circuit systems due to the light detection event of the generated associated direct macropixel and decrementing the corresponding counter due to the light detection event of the generated global macropixel, and simultaneously acquiring a plurality of direct signal contributions, each direct signal contribution being acquired using a different direct macropixel, wherein each direct macropixel is associated with one of the plurality of counter circuit systems.

[0175] (15) According to the method of (13) or (14), the histogram is also updated based on the output of the counter circuit system.

[0176] (16) The method of (15) also includes resetting the counter circuitry for each new bin of the histogram.

[0177] (17) The method according to any one of (11) to (16), wherein the direct macropixel and the global macropixel correspond to a plurality of adjacent light detection pixels among a plurality of light detection pixels.

[0178] (18) According to the method of (17), each of the plurality of light detection pixels includes a single-photon avalanche diode.

[0179] (19) The method according to any one of 11, wherein the indirectly reflected illumination light is caused by scattering or crosstalk of the directly reflected illumination light, and specifically, wherein the contribution of the multipath interference signal also includes the contribution from crosstalk within the sensor.

[0180] (20) A method for a LiDAR system, comprising: Illuminate the scene using modulated light; The reflected illumination light is imaged onto the LiDAR sensor; In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information. These direct signal contributions are obtained using directly reflected illumination light incident on direct macropixels. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is acquired, which is obtained by using the global macropixel incident with the indirectly reflected illumination light.

Claims

1. A LiDAR sensor configured as follows: In the region of interest within the active illumination portion of the LiDAR sensor, direct signal contributions are acquired to obtain distance information, wherein... The direct signal contribution is obtained by using the direct reflection of the illumination light incident on the direct macropixel; as well as In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is obtained, wherein the contribution of multipath interference signal is obtained by using the global macropixel incident on the indirectly reflected illumination light.

2. The LiDAR sensor according to claim 1, wherein, The direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

3. The LiDAR sensor of claim 1, further comprising a counter circuit system configured to increment due to light detection events of the generated direct macropixels and decrement due to light detection events of the generated global macropixels.

4. The LiDAR sensor of claim 1, comprising a plurality of counter circuitry systems, each counter circuitry system configured to increment due to generated photodetection events of associated direct macropixels and decrement due to generated photodetection events of said global macropixels, wherein, The LiDAR sensor is also configured to acquire multiple direct signal contributions simultaneously, each direct signal contribution being acquired using a different direct macropixel, wherein each direct macropixel is associated with one of the multiple counter circuit systems.

5. The LiDAR sensor of claim 3 is further configured to update the histogram based on the output of the counter circuitry system.

6. The LiDAR sensor according to claim 5, wherein, The counter circuitry is also configured to reset for each new bin in the histogram.

7. The LiDAR sensor according to claim 1, comprising a plurality of light detection pixels, each light detection pixel comprising a single-photon avalanche diode.

8. The LiDAR sensor according to claim 7, wherein, The direct macropixel and the global macropixel each correspond to a plurality of adjacent light detection pixels among the plurality of light detection pixels.

9. The LiDAR sensor according to claim 1, wherein, The indirectly reflected illumination light is caused by the scattering or crosstalk of the directly reflected illumination light. Specifically, the contribution of the multipath interference signal also includes the contribution from crosstalk within the sensor.

10. A LiDAR system, comprising: The illumination device is configured to illuminate the scene using modulated light; The imaging unit is configured to image the reflected illumination light onto the LiDAR sensor; as well as The LiDAR sensor is configured as follows: In the region of interest within the active illumination portion of the LiDAR sensor, a direct signal contribution is acquired to obtain distance information. This direct signal contribution is obtained using the direct macropixels to which the directly reflected illumination light is incident. In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is obtained, wherein the contribution of multipath interference signal is obtained by using the global macropixel incident on the indirectly reflected illumination light.

11. A method for a LiDAR sensor, comprising: In the region of interest within the active illumination portion of the LiDAR sensor, a direct signal contribution is acquired to obtain distance information, wherein the direct signal contribution is obtained using the direct macropixel incident on the directly reflected illumination light; and In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is obtained, wherein the contribution of multipath interference signal is obtained by using the global macropixel incident on the indirectly reflected illumination light.

12. The method according to claim 11, wherein, The direct signal contribution and the multipath interference signal contribution are acquired simultaneously.

13. The method of claim 11, further comprising incrementing the counter circuitry due to the generated light detection events of the direct macropixels, and decrementing the counter due to the generated light detection events of the global macropixels.

14. The method of claim 11, further comprising incrementing each of the plurality of counter circuit systems due to the generated associated photodetection event of the direct macropixel and decrementing the corresponding counter due to the generated photodetection event of the global macropixel, and simultaneously acquiring a plurality of direct signal contributions, each direct signal contribution being acquired using a different direct macropixel, wherein, Each direct macropixel is associated with one of the plurality of counter circuit systems.

15. The method of claim 13, further comprising updating the histogram based on the output of the counter circuit system.

16. The method of claim 15, further comprising resetting the counter circuitry for each new bin of the histogram.

17. The method according to claim 11, wherein, The direct macropixel and the global macropixel each correspond to a plurality of adjacent light detection pixels among a plurality of light detection pixels.

18. The method according to claim 17, wherein, Each of the plurality of light-detecting pixels includes a single-photon avalanche diode.

19. The method according to claim 11, wherein, The indirectly reflected illumination light is caused by the scattering or crosstalk of the directly reflected illumination light. Specifically, the contribution of the multipath interference signal also includes the contribution from crosstalk within the sensor.

20. A method for a LiDAR system, comprising: Illuminate the scene using modulated light; The reflected illumination light is imaged onto the LiDAR sensor; In the region of interest within the active illumination portion of the LiDAR sensor, a direct signal contribution is acquired to obtain distance information, wherein the direct signal contribution is obtained using the direct macropixel incident on the directly reflected illumination light; and In the region of interest outside the active illumination portion of the LiDAR sensor, the contribution of multipath interference signal is obtained, wherein the contribution of multipath interference signal is obtained by using the global macropixel incident on the indirectly reflected illumination light.

Images