Time-of-flight rising edge adaptive crosstalk correction

Abstract

The invention relates to time-of-flight rising edge adaptive crosstalk correction. A method of operating a time of flight (ToF) ranging system includes receiving a histogram including crosstalk signals generated by reflected light pulses from a cover glass of the ToF ranging system; finding a first rising edge in a first region of the histogram, the first rising edge having a gradient greater than a threshold or being a maximum gradient in the first region, where the first rising edge is in a first histogram box having a first value; determining a second value of a second histogram box in the first region, wherein the first histogram box precedes the second histogram box by a predetermined distance; estimating a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scaling the pre-stored light pulse shape by the estimated ratio; and subtracting the scaled pre-stored light pulse shape from the histogram.

Description

Technical Field

[0001] The present invention relates generally to a time-of-flight (ToF) ranging system, and in a particular embodiment, to a ToF ranging system capable of adaptively removing crosstalk signals from a histogram for better target detection and estimation. Background Art

[0002] Time-of-flight (ToF) imagers (also called ToF sensors) have recently been widely used in various applications such as gesture / facial recognition, light detection and ranging (LiDAR), virtual reality, augmented reality, and autonomous robots. ToF ranging systems use ToF imagers to measure the distance of an object (e.g., a target). To measure an object, the ToF ranging system instructs the ToF sensor to send a light signal (e.g., a light pulse) to the object and measures the time it takes for the signal to travel to and from the object.

[0003] A single photon avalanche diode (SPAD) can be used as a detector of reflected light. In some applications, a SPAD array is provided as a sensor (e.g., a SPAD array) to detect reflected light pulses. The reflected photons can generate carriers in the SPAD by the photoelectric effect. The photon-generated carriers can trigger an avalanche current in one or more SPADs in the SPAD array. The avalanche current can signal an event, i.e., a photon has been detected. Information related to the reflection intensity (also referred to as a "signal count") is output as a histogram of the SPAD array. The histogram of each SPAD includes a plurality of histogram bins, wherein each histogram bin corresponds to a distance (or a narrow range of distances) from the SPAD array, and the value of each histogram bin (e.g., a signal count) corresponds to the number of avalanche current events detected (e.g., the number of photons detected).

[0004] The histogram of the SPAD needs to be processed to extract useful information, such as the number of detected targets, the distance of the targets, etc. There are still challenges in the processing of the histogram. For example, light pulses reflected from the cover glass of the ToF ranging system generate crosstalk signals in the histogram. Before target detection and estimation can be performed reliably, the crosstalk signals need to be removed from the histogram. In addition, the processing of the histogram is usually computationally intensive and is usually performed by an off-chip processor located in a semiconductor die different from the semiconductor die with the SPAD array. Efficient processing of the histogram of the SPAD is required to reduce the computational complexity and achieve a higher level of integration of the ToF imager and / or ToF ranging system. Summary of the invention

[0005] In one embodiment, a method of operating a time-of-flight (ToF) ranging system includes: receiving a histogram generated by a ToF imager of the ToF ranging system, wherein the histogram includes a crosstalk signal corresponding to a reflected light pulse from a cover glass of the ToF ranging system; searching for a first rising edge having a first gradient in a first region of the histogram, wherein the first gradient is greater than a first threshold or is a maximum gradient in the first region, wherein the first rising edge is in a first histogram bin having a first value; determining a second value of a second histogram bin in the first region, wherein the first histogram bin is a predetermined distance before the second histogram bin; estimating a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scaling the pre-stored light pulse shape by the estimated ratio; and subtracting the scaled pre-stored light pulse shape from the histogram.

[0006] In one embodiment, a method of operating a time-of-flight (ToF) ranging system includes: generating a histogram by a ToF sensor of the ToF ranging system, wherein the histogram includes a crosstalk signal generated by a reflected light signal from a cover glass of the ToF ranging system; and removing the crosstalk signal from the histogram, including: estimating a gradient of a histogram bin in a first region of the histogram by calculating differences between adjacent histogram bins; finding a first rising edge in a first histogram bin in the first region of the histogram, wherein the first histogram bin has a first value and the gradient of the first histogram bin is greater than a threshold or is a maximum gradient in the first region; determining a second value of a second histogram bin in the first region, wherein the second histogram bin is a predetermined number after the first histogram bin; calculating a scaling factor of a pre-stored pulse shape based on the first value and the second value; scaling the pre-stored pulse shape by the scaling factor; and subtracting the scaled pre-stored pulse shape from the histogram.

[0007] In one embodiment, a time-of-flight (ToF) ranging system includes: a ToF sensor configured to emit light pulses and generate a histogram based on reflected light pulses; and a processor coupled to the ToF sensor, wherein the processor is configured to: search for a rising edge having a gradient in a first region of the histogram, wherein the gradient is greater than a threshold or is a maximum gradient in the first region, wherein the rising edge is in a first histogram bin having a first value; determine a second value of a second histogram bin in the first region, wherein the first histogram bin precedes the second histogram bin by a predetermined number; estimate a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scale the pre-stored light pulse shape by the estimated ratio; and subtract the scaled pre-stored light pulse shape from the histogram. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Details of one or more embodiments of the invention are set forth in the following drawings and description. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims. In the drawings, the same or similar reference numerals generally denote the same or similar components in the various drawings, and for the sake of brevity, these components will generally not be re-described. For a more complete understanding of the invention, reference is now made to the following description taken in conjunction with the drawings, in which:

[0009] Figure 1 A time-of-flight (ToF) ranging system in one embodiment is illustrated;

[0010] Figure 2 illustrates a histogram generated by a ToF imager in one embodiment;

[0011] Figure 3 A flowchart of a crosstalk cancellation method in one embodiment is illustrated;

[0012] Figure 4 An embodiment of the invention is shown in FIG. Figure 3 Histogram processed by the crosstalk elimination method;

[0013] Figure 5 illustrates a pre-stored light pulse in one embodiment;

[0014] Figure 6A-6C FIG. 1 shows an embodiment of the present invention. Figure 3 The performance of the crosstalk cancellation method;

[0015] Figure 7A-7B Another embodiment is shown in FIG. Figure 3 The performance of the crosstalk cancellation method;

[0016] Figure 8 The use of an embodiment is illustrated Figure 3 The performance of the ToF ranging system based on the crosstalk cancellation method;

[0017] Fig. 9 An integrated circuit (IC) device formed as a ToF ranging system in one embodiment is illustrated; and

[0018] Fig.10 A flow chart of a method of operating a ToF ranging system is illustrated. DETAILED DESCRIPTION

[0019] The manufacture and use of the currently preferred embodiments are discussed in detail below. However, it should be appreciated that the present invention provides many applicable inventive concepts that can be embodied in a variety of specific contexts. The specific embodiments discussed only illustrate specific ways to manufacture and use the present invention, and do not limit the scope of the present invention.

[0020] The present invention will be described in the context of a time-of-flight (ToF) ranging system, and in a particular embodiment, will be described in the context of a ToF ranging system capable of adaptively removing crosstalk signals from a histogram for better target detection and estimation.

[0021] Figure 1 A time of flight (ToF) ranging system 100 in one embodiment is illustrated. Note that for simplicity, not all features of the ToF ranging system 100 are illustrated. In addition, for ease of discussion, Figure 1 An object 105 (eg, a target) and environmental sources 102 are illustrated, but it should be understood that the object 105 and environmental sources 102 are not part of the ToF ranging system 100 . Figure 1 The upper part of FIG. 1 shows a cross-sectional view of the ToF ranging system 100 , and Figure 1 The lower portion of the diagram illustrates details of some components of the ToF ranging system 100, such as a top view of the SPAD array 101 and the reference SPAD array 101R, output pulses (e.g., indicating an avalanche current event) generated by the SPADs in the SPAD array and the reference SPAD array 101R, an OR tree for combining the output pulses, and details of the processing unit 107.

[0022] like Figure 1 As shown, the ToF ranging system 100 includes a SPAD array 101, a reference SPAD array 101R, a light source 103 (also referred to as a transmitter), and a processing unit 107 attached to a substrate 109 (eg, a printed circuit board (PCB), an interposer, etc.). Figure 1 An assembly housing 115 (e.g., a protective case, a box, a housing, etc.) is further illustrated. The SPAD array 101, the reference SPAD array 101R, the light source 103, the processing unit 107, and the substrate 109 are located in a space surrounded by the assembly housing 115. A cover glass 113 is attached to an upper surface of a side wall of the assembly housing 115, thereby forming a closed space to protect the components disposed in the space. Figure 1 The illustrated attachment of cover glass 113 to assembly housing 115 is a non-limiting example only. Cover glass 113 may be attached by any suitable method.

[0023] The light source 103 is a circuit configured to generate a light signal 104 for illuminating an object 105 (e.g., a target). The light source 103 may be, for example, a vertical cavity surface emitting laser (VCSEL) device, a light emitting diode (LED), an infrared (IR) device, etc. The light signal 104 may include a plurality of light pulses. A reflected light signal 106 from the object 105 is received by the SPAD array 101. The SPAD array 101 includes a plurality of SPADs arranged, for example, in rows and columns. Figure 1In the example of , the reference SPAD array 101R receives a reference light signal 104R from the light source 103 and provides a reference SPAD output. The reference SPAD output and the SPAD output (e.g., the output of the SPAD array 101) are sent to a ToF histogram generation circuit 107A to generate a histogram (also referred to as a ToF histogram). Methods for generating histograms using SPAD arrays and reference SPAD arrays are known in the art and are not discussed in detail here. For example, US20170115381 discloses circuits and methods for generating histograms using SPAD arrays and reference SPAD arrays. US20170115381 is incorporated herein by reference.

[0024] exist Figure 1 In the example of , the ToF histogram generation circuit 107A is part of the processing unit 107. The processing unit 107 also includes a processor 107B, which receives and processes the histogram. The processor 107B can be, for example, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), etc. The processor 107B processes the histogram to extract useful information, such as the number of (multiple) targets in the histogram, the distance of each target, etc. In an example embodiment, the processor 107B is configured to perform the method disclosed below to remove the crosstalk signal from the histogram, and can also be configured to use the histogram to estimate the distance of one or more targets after removing the crosstalk signal. In some embodiments, the ToF histogram generation circuit 107A and the processor 107B are integrated in the same semiconductor die (e.g., as a single integrated circuit (IC) die). In other embodiments, the ToF histogram generation circuit 107A and the processor 107B are formed in different (e.g., separate) semiconductor dies. These and other changes are fully intended to be included in the scope of the present disclosure.

[0025] Figure 1 A crosstalk signal 106C is illustrated, which represents an optical signal reflected by the cover glass 113 and received by the SPAD array 101. The crosstalk signal 106C is received by the SPAD array 101 and generates a corresponding pulse area in the histogram. In the discussion herein, the pulse area in the histogram corresponding to the crosstalk signal 106C (e.g., generated by the crosstalk signal 106C) is referred to as the crosstalk signal in the histogram, or simply the crosstalk signal. In order to correctly and reliably perform target detection and estimation, it is necessary to remove the crosstalk signal in the histogram. A crosstalk elimination method is disclosed below.

[0026] In some embodiments, the components for transmitting light signals, receiving reflected light signals, and generating histograms (including light source 103, SPAD array 101, reference SPAD array 101R, and ToF histogram generation circuit 107A) are collectively referred to as a ToF imager (also referred to as a ToF sensor). Therefore, the ToF ranging system 100 includes a ToF imager and a processor 107B, and the processor 107B processes the histogram generated by the ToF imager.

[0027] Figure 1 The light source 103, SPAD array 101, reference SPAD array 101R and processing unit 107 are illustrated as separate components attached to substrate 109. This is merely a non-limiting example. In some embodiments, the light source 103, SPAD array 101, reference SPAD array 101R and ToF histogram generation circuit 107A are integrated into a single IC die as an integrated ToF imager. In some embodiments, the light source 103, SPAD array 101, reference SPAD array 101R, ToF histogram generation circuit 107A and processor 107B are integrated into a single IC die as an integrated ToF ranging system. These and other variations are fully intended to be included within the scope of the present disclosure.

[0028] Figure 2 A histogram generated by a ToF imager in one embodiment is illustrated. Figure 2 The histogram in the example has multiple boxes (also called histogram boxes). The value of each box of the histogram (e.g., along the y-axis) represents the signal count within the duration of the box (e.g., a fixed duration). The position (e.g., along the x-axis) or index number of each box of the histogram corresponds to the distance (or a narrow range of distances) from the SPAD array 101. During the histogram processing, when a target is identified as being located in a histogram box, the distance of the histogram box is used as the distance of the target (e.g., the distance between the SPAD array 101 and the target). In the discussion herein, it is assumed that the light source 103 and the SPAD array 101 are located at the same location, so that the distance between the SPAD array 101 and the target is considered to be the same as the distance between the light source 103 and the target.

[0029] For ease of discussion, each bin of the histogram is assigned a distance (e.g., a distance corresponding to the center of the bin), and the distance corresponding to the bin is also referred to as the bin distance or bin distance. The bin index (also referred to as bin index or bin number) can be used to indicate the bin distance. For example, assuming that the first bin of the histogram (e.g., the leftmost bin or the earliest bin) has an index of 1, the second bin has an index of 2, and so on, the distance of the i-th bin can be calculated as ((i-1)+0.5)D BIN , where D BINis the distance traveled by the light signal in a duration equal to half the time interval T, and the time interval T is the duration of the histogram bin. The time interval T is also called the bin time width, and the distance D BIN Also called box width D BIN Note that the time interval T represents the round trip time of the optical signal. Figure 2 In the example of , the histogram includes two pulse regions. Pulse region 201 is generated by (eg, corresponds to) the crosstalk signal 106C, and pulse region 203 is generated by (eg, corresponds to) the reflected light signal from the target. Figure 2 In the example of , the peak of the pulse region 201 is located at the histogram bin with bin index Z. The distance of the histogram bin with bin index Z represents the distance between the ToF emitter and the cover glass, and the histogram bin with bin index Z is called the reference zero histogram bin. In general, given a specific design of the ToF ranging system 100, the distance D between the ToF emitter and the cover glass is CG is known, and therefore, the position (eg, bin number) of the reference zero histogram bin can be based on the distance D CG And box width D BIN (For example, D CG / D BIN ) to estimate.

[0030] In order to reliably detect the number of targets in the histogram and / or estimate the distance of (multiple) targets in the histogram, it is necessary to remove the crosstalk signal in the histogram. Otherwise, the crosstalk signal in the histogram may be mistakenly identified as a target, or may merge (e.g., partially merge or completely merge) with the pulse area corresponding to the target in the histogram and distort the pulse area. The distorted pulse area may cause errors or inaccuracies in target detection and estimation.

[0031] Typically, the crosstalk signal is removed from the histogram with a further safety margin based on a worst-case scenario. For example, for a specific ToF ranging system, during a characterization or calibration process, the reflection from the specific cover glass used is measured for a sample size of the cover glass. Based on the measurement, the worst-case scenario of the observed crosstalk signal is identified, and a cancellation scheme is designed to remove (e.g., eliminate) the worst (e.g., strongest) crosstalk signal observed during the calibration process, for example, by subtracting a known light pulse shape with a fixed amplitude from the histogram, where the fixed amplitude is selected based on the strongest crosstalk signal observed during the calibration process. When the actual crosstalk signal in the histogram is not the worst-case scenario, such cancellation schemes typically result in severe overcorrection, resulting in nonlinearity in a short range of the histogram. Another problem with existing crosstalk cancellation schemes is that the correction schemes are static and cannot correct crosstalk beyond the design limit or adapt to the individual conditions of each cover glass. During the service life of the ToF ranging system, debris or smudges may be applied to the cover glass during operation and may change dynamically (e.g., smudges are applied to the cover glass and cleaned a few minutes later). Cover glass may be scratched or degraded over time, changing its reflectivity over time. The calibration process is done only once, and the measurements may not be representative of the condition of another batch of cover glass. A method for dynamically removing crosstalk signals from the histogram based on the actual histogram can address the above challenges. Figure 3-Figure 5 Discuss example embodiments.

[0032] Figure 3 A flow chart of a crosstalk cancellation method 300 in one embodiment is shown. The discussion of the method 300 herein refers to Figure 4 and Figure 5 , Figure 4 and Figure 5 Picture shows Figure 3 The histogram and some concepts used in the method 300. It should be understood that Figure 3 The embodiment method shown is only one example of many possible embodiments. Those skilled in the art will recognize many variations, substitutions, and modifications. For example, the invention may be added, removed, replaced, rearranged, or repeated. Figure 3 The various steps shown.

[0033] refer to Figure 3 At block 301, a position a (e.g., bin index a) of a first rising edge is searched in a predetermined region of the histogram H. The predetermined region of the histogram H includes histogram bins around (e.g., before and after) a reference zero histogram bin. The bin number of the reference zero histogram bin is denoted as M, and the predetermined region may be a bin number between MN and MN. 1 To M+N 2 The histogram bins between N 1 and N2 is a positive number, such as 4. 1 and N 2 In some embodiments, assuming that the bin number of the first histogram bin is 1, the predetermined area may be the bin number between 1 and M+N. 2 For ease of discussion and without loss of generality, the boundaries of a predetermined region of the histogram H (eg, a start bin index and a stop bin index) are denoted as bin index Start and bin index Stop, respectively.

[0034] In some embodiments, the first rising edge in the predetermined region is the leftmost (e.g., earliest arriving) rising edge in the predetermined region whose gradient is greater than a predetermined threshold TH, or the maximum gradient in the predetermined region. The gradient of the first rising edge is indicated by the value of the corresponding histogram bin (e.g., having the same bin index a) in the difference histogram DH, where the difference histogram DH is generated from the histogram H.

[0035] In some embodiments, the difference histogram DH is generated by calculating the difference between adjacent bins of the histogram H, and the value of the difference histogram DH indicates the gradient at the bin of the histogram H. In an exemplary embodiment, the difference histogram is generated by calculating the difference between adjacent bins of the histogram and assigning the calculated difference to the corresponding bin of the difference histogram. For example, the value of the i-th bin of the histogram is represented as H(i), and the value of the i-th bin of the difference histogram is represented as DH(i), then DH(i) can be calculated as DH(i)=H(i)-H(i-1). In some embodiments, assuming that the index of the histogram bin starts from 1, a zero value is assigned to DH(1), so that the difference histogram DH has the same length as the histogram H (e.g., the same number of histogram bins), and there is a one-to-one correspondence between the histogram bins of the histogram H and the histogram bins (also referred to as bins) of the difference histogram DH. The value DH(i) of the i-th bin of the difference histogram DH is an approximation of (eg, proportional to) the gradient of the i-th bin of the histogram H and is therefore used herein as the gradient of the i-th bin of the histogram H.

[0036] In some embodiments, in order to find the first rising edge in a predetermined area of ​​the histogram H, the maximum value MaxVal of the difference histogram DH between the bin index Start and the bin index Stop is found, and the bin index of the first bin (e.g., the leftmost bin) of the difference histogram DH in the predetermined area and having the maximum value MaxVal is used as the position of the first rising edge.

[0037] In other embodiments, the position of the first rising edge is determined by finding a first box (e.g., the leftmost box) in a predetermined region of the histogram H that has a gradient greater than a predetermined threshold TH (as indicated by the value of the corresponding box of the difference histogram DH), the threshold TH being determined by the ambient noise level and a user-defined confidence level (e.g., a user-defined scaling factor). For example, to find the position of the first rising edge, the values ​​of the boxes of the difference histogram DH in the first pulse region (e.g., the box index is between Start and Stop) are compared with the threshold TH, and the first box (e.g., the leftmost box) in the first pulse region of the histogram H having a gradient value greater than the threshold TH is identified. The box index of the identified first box is used to indicate the position (or equivalently, the distance) of the first rising edge in the first pulse region of the histogram.

[0038] As an example, the threshold TH may be determined as follows. Denote the ambient event (eg, avalanche current event caused by an ambient light signal) of each histogram bin as σ 2 , and denote the user-defined confidence level as C, then the threshold TH is calculated as Note that in the above calculations, the environmental events for each bin of the differential histogram are considered to be twice the environmental noise events for each bin of the histogram (due to the subtraction operation used to calculate the differential histogram doubling the power of the environmental event level). Due to the shot noise characteristics of light, the noise in each differential histogram bin is the square root of twice the environmental events in each bin of the histogram. The environmental noise events for each histogram bin can be pre-measured using a calibration process, or can be measured periodically during operation of the ToF ranging system. The user-defined confidence level C can be adjusted according to the environmental noise level to achieve reliable detection of the first rising edge. For example, if the environmental noise level increases, the user-defined confidence level C can be increased, or vice versa, to avoid erroneous detection of the first rising edge and achieve the target detection rate of the first rising edge. In addition to the above method, other suitable methods for determining the threshold TH are also possible and are fully intended to be included within the scope of the present disclosure.

[0039] Figure 4 A histogram having a pulse region 401 and a pulse region 403 is illustrated. The pulse region 401 corresponds to the crosstalk signal, and the pulse region 403 corresponds to the reflected light signal from the target. Note that Figure 4 In , the histogram is illustrated as a curve rather than a bar graph, the curve being the envelope of the histogram bins, as the skilled person will readily appreciate. Figure 4In the histogram of , for simplicity, only the portion of the histogram containing pulses corresponding to reflections from the cover glass and the target (also referred to as a received pulse area or a pulse area) is illustrated, while other portions of the histogram (such as portions containing random noise (e.g., shot noise)) may not be plotted.

[0040] exist Figure 4 In the example of , the predetermined region is a portion of the histogram between bin indexes Start and Stop, which covers the region where the rising edge of the pulse region 401 and the reference zero histogram bin are located. Using, for example, the maximum gradient criterion or the threshold TH criterion discussed above, the position of the first rising edge is found to be in the histogram bin with bin index a.

[0041] Reference again Figure 3 At block 303, a histogram value H(b) at bin index b is obtained (e.g., b=a+D). In some embodiments, bin index b is a predetermined number D after bin index a. In some embodiments, the predetermined number D is at a predetermined offset D. OFF Between about 50% and about 100% (e.g., 0.5×D OFF <D<D OFF ), such as a predetermined offset D OFF In the exemplary embodiment, the predetermined offset D OFF is half the width of a light pulse (measured in units of distance) transmitted by the light source 103 of the ToF ranging system 100. For example, the duration of the transmitted light pulse is represented by T P , and let C be the speed of light, then when measured in meters, D OFF =0.5×T P ×C, or when measured in the number of histogram bins, D OFF =0.5×T P ×C / D BIN .exist Figure 4 In the example, with bin index a+D OFF The histogram bin of corresponds to the center of the pulse region 401 and has the maximum value (also called peak value) of the pulse region 401 (eg, the crosstalk signal).

[0042] Note that the predetermined offset D OFF The predetermined offset D is determined by the shape of the transmitted light pulse generated by the light source 103 (eg, a VCSEL device), which shape may be characterized (eg, measured) in advance, for example, by a calibration process of the light source 103. In some embodiments, the predetermined offset D OFF is predetermined, saved and used later, for example, to determine the predetermined number D and / or to calculate the distance to the target. Note that due to the predetermined offset D OFFIndependent of the various factors that distort the shape of the received pulse area in the histogram, the predetermined offset D OFF The predetermined number D and the estimate of the target distance are robust and not affected by the received pulse shape distortion.

[0043] The histogram value H(b) at bin index b is used in subsequent processing to calculate the ratio R between the crosstalk signal (e.g., pulse region 401) and the pre-stored optical pulse shape. In some embodiments, the pre-stored optical pulse shape (e.g., see Figure 5 ) is a histogram of transmitted light pulses acquired during a calibration process, for example, when the transmitted light pulses are reflected by a single target with little or no ambient noise. In other words, the pre-stored light pulse shape represents an ideal histogram of the transmitted light pulses and is very similar to the shape of the transmitted light pulses. In the illustrated embodiment, the pre-stored light pulse shape is a normalized histogram of the transmitted light pulses such that the maximum value of the pre-stored light pulse shape is a unit value of 1. The pre-stored light pulse shape may be stored in a memory area of ​​the ToF ranging system (e.g., see Fig. 9 1015 in).

[0044] The crosstalk signal in the histogram (e.g., pulse region 401) may ideally be a scaled version of a pre-stored optical pulse shape. Thus, by finding, for example, a ratio R between the maximum value of the crosstalk signal and the maximum value of the pre-stored optical pulse shape, the pre-stored optical pulse shape may be scaled by the ratio R (e.g., multiplied by the ratio R) and then subtracted from the histogram to generate a crosstalk-free (or substantially crosstalk-free) histogram.

[0045] However, during operation of the ToF ranging system, the crosstalk signal in the histogram may be distorted due to various factors, such as due to a target located at a very close distance (e.g., a few millimeters) from the cover glass. Such a close target generates a pulse area in the histogram, which can merge with the pulse area caused by the cover glass, thereby distorting the pulse area (e.g., 401) containing the crosstalk signal. If the maximum value of the pulse area 401 is used to calculate the ratio R, the close target may cause the maximum value of the pulse area to be exaggerated and result in an erroneously large ratio R. By using the histogram value H(b) at bin index b, where bin index b=a+D, the ratio of the position a+D is calculated. OFFCloser to the first rising edge, the disclosed method 300 reduces the chance of a close target interfering with the estimation of the ratio R. This is because the reflected light signal from the close target arrives at the SPAD array later than the reflected light from the cover glass, and therefore, the pulse region corresponding to the close target does not distort the beginning portion of the pulse region corresponding to the cover glass (referred to as the rising edge region). In other words, the rising edge region of the pulse region corresponding to the cover glass (e.g., 401) is not distorted by the close target and can be used to provide a reliable estimation of the ratio R. More details will be disclosed below.

[0046] refer to Figure 4 , the first rising edge of pulse region 401 is located at bin index a. The value of the histogram bin at bin index b is H(b). To obtain bin index c=a+D OFF The value H(b) is scaled by a predetermined value S. In other words, PeakEst=S×H(b). The predetermined value S can be determined based on the known shape of the transmitted light pulse. The details will be referred to below. Figure 5 Let's discuss.

[0047] Figure 5 A pre-stored light pulse shape (e.g., a bar histogram) is illustrated, which in one embodiment is a histogram of emitted light pulses acquired, for example, during a calibration process. The bar histogram is very similar to the shape of the light pulses emitted by the light source 103 of the ToF ranging system. In some embodiments, the pre-stored light pulse shape has the same timing resolution (e.g., bin time width) as the histogram generated by the ToF ranging system 100 during normal operation, and corresponds to histogram bins around the reference zero histogram bin (e.g., bins between bin index 1 to M+K, where M is the bin index of the reference zero histogram and K is a positive number, such as 4, 5, or greater). The pre-stored light pulse shape can be analyzed according to the same process as described above to find the position a of the first rising edge in the pre-stored light pulse shape. The predetermined value S can be calculated by calculating the bin index c (e.g., c=a+D OFF ) and b (e.g., b=a+D). In other words, S=PH(c) / PH / (b), where PH(c) and PH(b) represent the values ​​of the pre-stored light pulse shapes at bin indexes c and b, respectively. In some embodiments, the predetermined value S is pre-calculated and saved for later use.

[0048] As described above, the maximum value of the crosstalk signal can be estimated by PeakEst=S×H(b). Since the maximum value of the pre-stored optical pulse shape (which is a normalized histogram) is 1, the ratio R between the crosstalk signal in the histogram and the pre-stored optical pulse shape can be calculated as R=PeakEst=S×H(b). Then, the pre-stored optical pulse shape can be scaled by the ratio R, and then, the scaled pre-stored optical pulse shape is subtracted from the histogram to generate a cleaner histogram with little or no crosstalk signal.

[0049] exist Figure 4 In the example of , the pulse region 401 corresponding to the crosstalk signal is separated from the pulse region 403, and the shape of the transmitted light pulse is maintained. However, during operation of the ToF ranging system 100, various factors may distort the pulse region 401. For example, random noise, the number of targets, the relative position of the targets, and the receiver bandwidth (e.g., the bandwidth of the SPAD array) may introduce severe distortion in the received pulse region and may cause errors in the calculated ratio R, or may even cause the processing discussed above to fail to detect the first rising edge. In order to handle various corner situations and prevent over-correction or under-correction, additional boundary checks and processing are incorporated into the method 300, and the details will be discussed below.

[0050] Reference again Figure 3 , at block 305, the histogram bin values ​​H(b) are compared to the histogram bin values ​​H(a). If H(b)>H(a), which is the expected case, processing proceeds to block 307. However, if H(b)≤H(a), this may indicate that the pulse region 401 is severely distorted and the histogram bin values ​​H(b) are not suitable for use in calculating the ratio R, and processing proceeds to block 309.

[0051] At box 307, the histogram bin value H(b) is compared with an upper bound UB, where the upper bound UB is the maximum allowed value of the histogram bin value H(b). If H(b)>UB, this may indicate that the value H(b) is unreliable and may be exaggerated, for example, by a close target, and therefore should not be used to calculate the estimated maximum value PeakEst of the crosstalk signal. Therefore, processing proceeds to box 317, where the estimated maximum value PeakEst is assigned a default value DefaultVal. However, if H(b) is not greater than the upper bound UB, processing proceeds to box 311, where the estimated maximum value PeakEst is calculated as PeakEst=S×H(b), where S=PH(c) / PH(b), as described above.

[0052] Next, after the processing at block 311, at block 315, the estimated maximum value PeakEst obtained in block 311 is compared with a lower bound LB, where the lower bound LB is the minimum allowed value of the estimated maximum value PeakEst. If the estimated maximum value PeakEst is less than the lower bound LB, this may indicate an error in the processing (e.g., the first rising edge is selected at the wrong position due to random noise), and therefore, the processing proceeds to block 317, where the estimated maximum value PeakEst is assigned a default value DefaultVal. If the estimated maximum value PeakEst is not less than the lower bound LB, the estimated maximum value PeakEst is not changed, and the processing proceeds to block 319, where the estimated maximum value PeakEst (which is the same as the ratio R since the pre-stored optical pulse shape is a normalized histogram) is used to scale the pre-stored optical pulse shape, and the scaled pre-stored optical pulse shape is subtracted from the histogram H to obtain a clean histogram with a reduced crosstalk signal.

[0053] If, in box 305, the value H(b) is not greater than the value H(a), processing proceeds to box 309, where the value H(a) is compared to an upper bound UB. If H(a) is greater than the upper bound UB, this may indicate an error, and processing proceeds to box 317, where the estimated maximum value PeakEst is assigned a default value DefaultVal. If H(a) is not greater than the upper bound UB, processing proceeds to box 313, where the estimated maximum value PeakEst is calculated as PeakEst=L×S×H(a), where S=PH(c) / PH(b), as described above, and L is a scaling factor greater than 1, such as 2. Next, processing proceeds to box 315, where the estimated maximum value PeakEst is compared to a lower bound LB. The subsequent processing in boxes 317 and 319 has been discussed previously, so the details will not be repeated.

[0054] In some embodiments, the upper bound UB and the lower bound LB can be determined by, for example, calibration, simulation, a combination thereof, etc. In some embodiments, the upper bound UB and the lower bound LB are programmable values ​​and are set with initial values ​​obtained by a calibration process. During the life of the ToF ranging system 100, the upper bound UB and the lower bound LB can be updated, for example, based on aggregated data about past performance, and can therefore be modified to improve the performance of the ToF ranging system 100.

[0055] The upper bound UB and the lower bound LB avoid over-correction and under-correction of the crosstalk signal. The default value DefaultVal of the estimated maximum value PeakEst allows a reasonable amount of elimination of the crosstalk signal in the histogram, even when the method fails to find the correct position of the rising edge due to, for example, excessive distortion or large noise in the pulse region 401. Figure 4 Examples of an upper bound UB, a lower bound LB, and a default value DefaultVal are shown.

[0056] In some embodiments, after crosstalk cancellation, the rising edge detection method discussed above is used to estimate the distance of the target in each pulse region of the histogram. Note that each pulse region in the histogram may include one target, or may include more than one closely-spaced target (referred to as a target cluster in the pulse region), in which case the reflected light signals from the target clusters merge together to form a long, distorted pulse region (e.g., distorted compared to the transmitted light pulse). However, as described above, the position and shape of the first rising edge (e.g., the leftmost rising edge) of each pulse region is generally not distorted and can be used to reliably estimate the distance of the first target (e.g., the closest target) contained in the pulse region.

[0057] In one embodiment, after performing crosstalk cancellation on the histogram, for each pulse region of the histogram, the first rising edge (e.g., the leftmost rising edge) is found by, for example, finding the first histogram bin (e.g., the leftmost histogram bin) that has a gradient greater than a threshold TH or is the largest gradient in the pulse region. Note that for target detection, the confidence level C (e.g., ) is different from the confidence level used for crosstalk cancellation. Once the location of the first rising edge in each pulse region is found (e.g., bin index), the predetermined offset D OFF The distance to the first rising edge is added to obtain the distance to the first target (e.g., the nearest target) in the pulse area. Additional details and additional methods of target detection and estimation based on rising edges are discussed in U.S. Patent Application No. 18 / 342,965, which is incorporated herein by reference. Any suitable method disclosed in U.S. Patent Application No. 18 / 342,965 can be used for target detection and estimation.

[0058] Figure 6A-6C FIG. 1 shows an embodiment of the present invention. Figure 3 The performance of the crosstalk cancellation method. Fig. 6A , curve 601 illustrates a pre-stored light pulse shape (eg, a normalized histogram with a maximum amplitude of 1). Figure 6B A histogram with two pulse regions 603 and 605 is shown. Figure 6BIn the example of FIG. 6 , pulse region 603 corresponds to a crosstalk signal, and pulse region 605 corresponds to a target. Pulse regions 603 and 605 are separate (eg, not merged). Figure 6C Shows the use of Figure 3 The histogram after the crosstalk signal is eliminated by the method. Figure 6C As shown, Figure 6B The pulse region 603 in FIG. 1 is removed, and the histogram after crosstalk cancellation includes only the pulse region 605 .

[0059] Figure 7A-7B Another embodiment is shown in FIG. Figure 3 The performance of the crosstalk cancellation method. Fig. 7A A histogram is shown with pulse region 613 and pulse region 615. Pulse region 613 corresponds to a crosstalk signal and pulse region 615 corresponds to a target. The target is very close to the cover glass, so that pulse region 615 overlaps (eg, merges) with pulse region 613. Figure 7A-7B Example of pre-stored pulse shapes with Fig. 6A are the same and therefore not shown again. Figure 7B Shown using Figure 3 The histogram after crosstalk elimination of the method. Figure 7B As shown, the crosstalk signal is removed from the histogram. Due to the overlap of the crosstalk signal 613 and the pulse area 615, the removal of the crosstalk signal generates some distortion (e.g., some notches) around the bin index 30. However, for the purpose of target detection and target distance estimation, these distortions do not cause performance degradation.

[0060] Figure 8 The use of an embodiment is illustrated Figure 3 The performance of the ToF ranging system based on the crosstalk cancellation method. Figure 8 In the figure, the x-axis represents the target distance and the y-axis represents the target distance reported by the ToF ranging system. Figure 3 The clean histogram (e.g., after crosstalk cancellation) is then used to estimate the distance of the target using the rising edge-based target detection and estimation method described above. Figure 8 In FIG. 8 , the dashed line 801 with a 45 degree angle represents the ideal performance (y=x). Curve 803 represents the performance of the Figure 3 The performance of the ToF ranging system using the crosstalk cancellation method is shown in Figure 8. For comparison, the performance of two other ToF ranging systems is described. In particular, curve 805 represents the ranging results of the ToF ranging system without using any crosstalk cancellation, and curve 807 represents the ranging results of the ToF ranging system using a fixed crosstalk cancellation scheme (e.g., using a fixed scaling factor for a pre-stored optical pulse shape).

[0061] like Figure 8 As shown, by adaptively estimating the scaling factor (e.g., ratio R) of the pre-stored optical pulse shape, using Figure 3 The ToF ranging system using the crosstalk cancellation method achieves the same performance as the ideal performance shown by the dashed line 801. In contrast, curves 805 and 807 show that the corresponding ToF ranging system without using the currently disclosed method fails at long distances. This may be due to the low target signal level at long distances, which causes the ToF ranging system to mistakenly identify the crosstalk signal as the target.

[0062] Fig. 9 An integrated circuit (IC) device 1000 formed as a ToF ranging system in one embodiment is shown. The IC device 1000 may be formed as Figure 1 ToF ranging system 100. Note that for simplicity, not all features of IC device 1000 are shown. IC device 1000 includes a light source 1013 (eg, VCSEL, LED, etc.) and a SPAD array 1010. Light source 1013 may correspond to Figure 1 The SPAD array 1010 may correspond to Figure 1 The SPAD array 101 and the reference SPAD array 101R may further include Figure 1 ToF histogram generating circuit 107A.

[0063] IC device 1000 also includes a memory module 1015 for storing histograms generated by SPAD array 1010. Memory module 1015 may also store pre-stored optical pulse shapes and high-resolution pulse shapes (see, e.g., Figure 5 ), predetermined value S, etc. It is worth noting that Fig. 9 In the embodiment, the processor 1017 is integrated in the IC device 1000, and the processor 1017 may correspond to Figure 1 The processor 107B in the IC device 1000 can be configured to perform one or more of the methods disclosed herein to perform crosstalk cancellation and generate an estimate of the distance of a target in the pulse area. The IC device 1000 can also include an input / output (I / O) module 1019 for communication with other devices.

[0064] Note that the various methods disclosed for target distance estimation are easy to implement and require very little computing power. These advantages allow many different types of processors to be easily and economically integrated into the IC device 1000 as the processor 1017, and no off-chip processing of the histogram by another processor is required. On-chip processing of the histogram data significantly reduces the chip complexity associated with the I / O transfer of the histogram data to the off-chip processing module, and reduces the processing delay associated with off-chip processing.

[0065] Variations and modifications to the disclosed method are possible and are fully intended to be included within the scope of the present disclosure. Figure 3 After finding the position of the first rising edge in the predetermined area in block 301 and before the processing in block 303, the adjustment item D ADJ Added to the position of the first rising edge to fine-tune the position of the first rising edge. Used to calculate the adjustment item D ADJ Different methods are possible. The following discussion is used to determine the adjustment term D ADJ Two different implementation methods.

[0066] In one embodiment, the adjustment term D is calculated using the value of the bin of the difference histogram DH at the location of the first rising edge found in block 301 and using the bin values ​​of the histogram H around the location of the first rising edge. ADJ For example, assuming that from block 301, the position of the first rising edge is determined to be in the mth bin of the histogram, then the adjustment term D ADJ Calculated as D ADJ =DH(m) / (H(m)+H(m+1)). Adjustment term D ADJ It is calculated by dividing DH(m) by the sum of H(m) and H(m+1). In other words, the adjustment term D ADJ The shape of the histogram around the first rising edge is taken into account and thus a value smaller than the bin width D can be provided. BIN The fine adjustment of the distance improves the estimation accuracy of the position (e.g., distance) of the first rising edge.

[0067] In another embodiment, the position of the first rising edge is fine-tuned by performing a fitting process between the first rising edge and a pre-stored high-resolution rising edge. The edge fitting process is described in U.S. Patent Application No. 18 / 342,965, and the details are not repeated here. The fitting process provides a sub-bin adjustment of the position of the first rising edge.

[0068] Note that the adjustment item D ADJ can be a fractional value, and the bin index b in block 303 should be changed to b=a+D+D ADJ To combine fine-tuning. To obtain the histogram value of H(b) for the fractional index number b, an interpolation process (e.g., linear interpolation or other interpolation methods) can be performed on the histogram H to generate histogram values ​​at positions between the integer number of histogram bins. In addition, to calculate the predetermined value S of the number of fractional bins (e.g., S=PH(c) / PH(b)), a high-resolution histogram of the emitted light pulses can be used, as described below with reference to Figure 5 described.

[0069] refer to Figure 5 , Figure 5The curve 501 in FIG. 5 is the envelope of the high-resolution histogram of the emitted light pulse, where the sampling rate of the high-resolution histogram is higher than Figure 5 The sampling rate of the bar histogram in is increased (e.g., 10 times, 100 times, or more), where the bar histogram is generated at the sampling rate used during normal operation of the ToF ranging system. The high resolution histogram shown in curve 501 can be obtained during a calibration process with a high sampling rate. The ratio between the high sampling rate of the high resolution histogram and the high sampling rate of the bar histogram is called the oversampling factor OV. Figure 5 The high-resolution histogram provides a sub-bin resolution compared to a bar histogram of , and the values ​​of the high-resolution histogram can be used to provide values ​​of the histogram at bins with a fractional number of bins, for example, by dividing the number of bins of the high-resolution histogram by the oversampling factor OV and finding the number that is closest to the fractional number of bins). Figure 5 Illustrated are example bin indices b and c as fractions, a high resolution histogram indicated by curve 501 may provide histogram values ​​at these fractional bin indices to calculate the predetermined value S.

[0070] Fig.10 A method 1100 of operating a ToF ranging system in one embodiment is illustrated. It should be understood that Fig.10 The embodiment method shown is only one example of many possible embodiments. Those skilled in the art will recognize many variations, substitutions, and modifications. For example, the invention may be added, removed, replaced, rearranged, or repeated. Fig.10 The various steps shown.

[0071] refer to Fig.10 , at box 1010, a histogram generated by a ToF imager of a ToF ranging system is received, wherein the histogram includes a crosstalk signal corresponding to a reflected light pulse from a cover glass of the ToF ranging system. At box 1020, a first rising edge having a first gradient is searched in a first region of the histogram, wherein the first gradient is greater than a first threshold or is a maximum gradient in the first region, wherein the first rising edge is in a first histogram bin having a first value. At box 1030, a second value of a second histogram bin in the first region is determined, wherein the first histogram bin is a predetermined distance before the second histogram bin. At box 1040, a ratio between the first region of the histogram and a pre-stored light pulse shape is estimated based on the first value and the second value. At box 1050, the pre-stored light pulse shape is scaled by the estimated ratio. At box 1060, the scaled pre-stored light pulse shape is subtracted from the histogram.

[0072] The disclosed embodiments can achieve advantages. For example, the disclosed method provides a method for adaptively eliminating (e.g., removing) crosstalk signals from a histogram. By dynamically estimating the ratio R using a histogram, an appropriate amount of crosstalk signals is removed from the histogram. The disclosed method automatically adjusts the amount of crosstalk signals to be removed based on the current condition of the cover glass (e.g., scratches, stains, dirt). In addition, the disclosed method also corrects for changes in the quality of the cover glass, changes in the thickness of the cover glass, changes in the air gap, etc., thereby achieving excellent crosstalk elimination without overcorrection. The upper and lower bounds introduced in method 300 help prevent overcorrection and undercorrection, and the default values ​​introduced ensure that at least a reasonable amount of crosstalk signals are removed, even when distortion in the histogram causes method 300 to be unable to find the correct position of the first rising edge. The disclosed method is easy to implement and saves computing power, and therefore, allows a ToF ranging system to be fully integrated on a single IC device.

[0073] Example embodiments of the present invention are summarized here. Other embodiments may also be understood from the overall description and claims submitted here.

[0074] Example 1. In one embodiment, a method of operating a time-of-flight (ToF) ranging system includes: receiving a histogram generated by a ToF imager of the ToF ranging system, wherein the histogram includes a crosstalk signal corresponding to a reflected light pulse from a cover glass of the ToF ranging system; searching for a first rising edge having a first gradient in a first region of the histogram, wherein the first gradient is greater than a first threshold or is a maximum gradient in the first region, wherein the first rising edge is in a first histogram bin having a first value; determining a second value of a second histogram bin in the first region, wherein the first histogram bin is a predetermined distance before the second histogram bin; estimating a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scaling the pre-stored light pulse shape by the estimated ratio; and subtracting the scaled pre-stored light pulse shape from the histogram.

[0075] Example 2. The method according to Example 1 also includes detecting a target in the histogram after the subtraction in the following manner: searching for a second rising edge with a second gradient in a second region of the histogram, wherein the second gradient is greater than a second threshold different from the first threshold, or is the maximum gradient in the second region; and calculating an estimate of the distance of the target by adding a predetermined offset to the distance of the second rising edge.

[0076] Example 3. The method according to Example 1 further includes, before finding the first rising edge: calculating the difference between adjacent histogram bins in the first region; and assigning the calculated difference as the gradient of the corresponding histogram bin in the first region.

[0077] Example 4. A method according to Example 1, wherein the first area includes a plurality of histogram bins around a reference zero histogram bin, wherein the distance of the reference zero histogram bins corresponds to the distance between a transmitter of the ToF ranging system and the cover glass of the ToF ranging system.

[0078] Example 5. A method according to Example 1, wherein estimating the ratio includes: comparing the second value with the first value; and in response to determining that the second value is greater than the first value: checking whether the second value is between an upper bound and a lower bound; and in response to determining that the second value is between the upper bound and the lower bound, generating the estimate of the ratio by multiplying the second value by a first predetermined scaling factor.

[0079] Example 6. The method of Example 5 further includes assigning a predetermined default value as the estimate of the ratio in response to determining that the second value is above the upper bound or below the lower bound.

[0080] Example 7. A method according to Example 6, wherein estimating the ratio further includes: in response to determining that the second value is less than the first value: checking whether the first value is between the upper bound and the lower bound; and in response to determining that the first value is between the upper bound and the lower bound, generating the estimate of the ratio by multiplying the first value by a second predetermined scaling factor.

[0081] Example 8. The method of Example 7 further comprises assigning the predetermined default value as the estimate of the ratio in response to determining that the first value is above the upper bound or below the lower bound.

[0082] Example 9. The method of Example 1 further comprises, after finding the first rising edge and before determining the second value: fine-tuning the position of the first rising edge by calculating an adjustment term and adding the adjustment term to the position of the first rising edge.

[0083] Example 10. A method according to Example 9, wherein the adjustment term is a fraction, and wherein determining the second value of the second histogram bin comprises: performing an interpolation process using the values ​​of the histogram bins around the second histogram bin to generate an interpolated histogram value; and assigning the interpolated histogram value as the second value.

[0084] Example 11. In one embodiment, a method of operating a time-of-flight (ToF) ranging system includes: generating a histogram by a ToF sensor of the ToF ranging system, wherein the histogram includes a crosstalk signal generated by a reflected light signal from a cover glass of the ToF ranging system; and removing the crosstalk signal from the histogram, including: estimating the gradient of a histogram bin in a first region of the histogram by calculating the difference between adjacent histogram bins; searching for a first rising edge in a first histogram bin in the first region of the histogram, wherein the first histogram bin has a first value and the gradient of the first histogram bin is greater than a threshold or is a maximum gradient in the first region; determining a second value of a second histogram bin in the first region, wherein the second histogram bin is a predetermined number after the first histogram bin; calculating a scaling factor of a pre-stored pulse shape based on the first value and the second value; scaling the pre-stored pulse shape by the scaling factor; and subtracting the scaled pre-stored pulse shape from the histogram.

[0085] Example 12. A method according to Example 11, wherein the first area includes a plurality of histogram bins around a reference zero point histogram bin, wherein the distance of the reference zero point histogram bins corresponds to the distance between a ToF transmitter of the ToF ranging system and the cover glass.

[0086] Example 13. The method of Example 11, wherein the ToF sensor is configured to emit a light signal comprising light pulses, wherein the pre-stored pulse shape corresponds to a normalized histogram of the light pulses emitted by the ToF sensor.

[0087] Example 14. The method of Example 13, wherein the scaling factor is a ratio between a third value in a third histogram bin in the first region of the histogram and a corresponding fourth value of the normalized histogram.

[0088] Example 15. A method according to Example 11, wherein calculating the scaling factor includes: determining whether the second value is greater than the first value; and in response to determining that the second value is greater than the first value: checking whether the second value is between an upper bound and a lower bound; and in response to determining that the second value is between the upper bound and the lower bound, calculating the scaling factor by multiplying the second value by a predetermined value.

[0089] Example 16. The method of Example 15, further comprising: in response to determining that the second value is outside the range between the upper bound and the lower bound, assigning a default value as the scaling factor.

[0090] Example 17. In one embodiment, a time-of-flight (ToF) ranging system includes: a ToF sensor configured to emit light pulses and generate a histogram based on reflected light pulses; and a processor coupled to the ToF sensor, wherein the processor is configured to: search for a rising edge with a gradient in a first region of the histogram, wherein the gradient is greater than a threshold or is a maximum gradient in the first region, wherein the rising edge is in a first histogram bin having a first value; determine a second value of a second histogram bin in the first region, wherein the first histogram bin is a predetermined number before the second histogram bin; estimate a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scale the pre-stored light pulse shape by the estimated ratio; and subtract the scaled pre-stored light pulse shape from the histogram.

[0091] Example 18. A ToF ranging system according to Example 17, wherein the first area includes a plurality of histogram bins around a reference zero point histogram bin, wherein the distance of the reference zero point histogram bins corresponds to the distance between a ToF transmitter of the ToF ranging system and a cover glass of the ToF ranging system.

[0092] Example 19. A ToF ranging system according to Example 18, wherein the pre-stored light pulse shape is a normalized histogram of light pulses emitted by the ToF sensor.

[0093] Example 20. A ToF ranging system according to Example 19, wherein the processor is further configured to: after the subtracting, use the histogram to estimate the distance of a target of the ToF ranging system.

[0094] Although the present invention has been described with reference to exemplary embodiments, this description should not be construed as having a limiting meaning. Various modifications and combinations of the illustrative embodiments and other embodiments of the present invention will be clear to those skilled in the art with reference to the specification. Therefore, the appended claims are intended to include any such modifications or embodiments.

Claims

1. A method of operating a time-of-flight (ToF) ranging system, the method comprising: receiving a histogram generated by a ToF imager of the ToF ranging system, wherein the histogram includes a crosstalk signal corresponding to a reflected light pulse from a cover glass of the ToF ranging system; searching for a first rising edge having a first gradient in a first region of the histogram, wherein the first gradient is greater than a first threshold or the first gradient is a maximum gradient in the first region, wherein the first rising edge is in a first histogram bin having a first value; determining a second value for a second histogram bin in the first region, wherein the first histogram bin precedes the second histogram bin by a predetermined distance; estimating a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scaling the pre-stored optical pulse shape by the estimated ratio; as well as A scaled pre-stored light pulse shape is subtracted from the histogram.

2. The method of claim 1 , further comprising detecting an object in the histogram after the subtraction by: searching for a second rising edge having a second gradient in a second region of the histogram, wherein the second gradient is greater than a second threshold value different from the first threshold value, or the second gradient is a maximum gradient in the second region; and An estimate of the distance to the target is calculated by adding a predetermined offset to the distance of the second rising edge.

3. The method according to claim 1, further comprising, before searching for the first rising edge: calculating differences between adjacent histogram bins in the first region; and The calculated difference is assigned as the gradient of the corresponding histogram bin in the first region.

4. The method of claim 1 , wherein the first region comprises a plurality of histogram bins around a reference zero histogram bin, wherein a distance of the reference zero histogram bins corresponds to a distance between a transmitter of the ToF ranging system and the cover glass of the ToF ranging system.

5. The method of claim 1 , wherein estimating the ratio comprises: comparing the second value to the first value; as well as In response to determining that the second value is greater than the first value: Checking whether the second value is between an upper bound and a lower bound; as well as In response to determining that the second value is between the upper bound and the lower bound, the estimate of the ratio is generated by multiplying the second value by a first predetermined scaling factor. 6 . The method of claim 5 , further comprising assigning a predetermined default value as the estimate of the ratio in response to determining that the second value is above the upper bound or the second value is below the lower bound.

7. The method of claim 6, wherein estimating the ratio further comprises: In response to determining that the second value is less than the first value: checking whether the first value is between the upper bound and the lower bound; as well as In response to determining that the first value is between the upper bound and the lower bound, the estimate of the ratio is generated by multiplying the first value by a second predetermined scaling factor. 8 . The method of claim 7 , further comprising assigning the predetermined default value as the estimate of the ratio in response to determining that the first value is above the upper bound or below the lower bound.

9. The method of claim 1 , further comprising, after finding the first rising edge and before determining the second value: The position of the first rising edge is fine-tuned by calculating an adjustment term and adding the adjustment term to the position of the first rising edge.

10. The method of claim 9, wherein the adjustment term is a fraction, wherein determining the second value of the second histogram bin comprises: performing an interpolation process using values ​​of said histogram bins surrounding said second histogram bin to generate interpolated histogram values; as well as The interpolated histogram value is assigned as the second value.

11. A method of operating a time-of-flight (ToF) ranging system, the method comprising: generating a histogram by a ToF sensor of the ToF ranging system, wherein the histogram includes a crosstalk signal generated by a reflected light signal from a cover glass of the ToF ranging system; as well as Removing the crosstalk signal from the histogram includes: estimating gradients of histogram bins in a first region of the histogram by computing differences between adjacent histogram bins; Finding a first rising edge in a first histogram bin in the first region of the histogram, wherein the first histogram bin has a first value and a gradient of the first histogram bin is greater than a threshold or a gradient of the first histogram bin is a maximum gradient in the first region; determining a second value for a second histogram bin in the first region, wherein the second histogram bin is a predetermined number after the first histogram bin; calculating a scaling factor for a pre-stored pulse shape based on the first value and the second value; scaling the pre-stored pulse shape by the scaling factor; and A scaled pre-stored pulse shape is subtracted from the histogram.

12. The method of claim 11, wherein the first region comprises a plurality of histogram bins around a reference zero histogram bin, wherein a distance of the reference zero histogram bins corresponds to a distance between a ToF transmitter of the ToF ranging system and the cover glass.

13. The method of claim 11, wherein the ToF sensor is configured to emit a light signal comprising light pulses, wherein the pre-stored pulse shape corresponds to a normalized histogram of the light pulses emitted by the ToF sensor.

14. The method of claim 13, wherein the scaling factor is a ratio between a third value in a third histogram bin in the first region of the histogram and a corresponding fourth value of the normalized histogram.

15. The method of claim 11, wherein calculating the scaling factor comprises: determining whether the second value is greater than the first value; as well as In response to determining that the second value is greater than the first value: Checking whether the second value is between an upper bound and a lower bound; as well as In response to determining that the second value is between the upper bound and the lower bound, the scaling factor is calculated by multiplying the second value by a predetermined value.

16. The method according to claim 15, further comprising: In response to determining that the second value is outside the range between the upper bound and the lower bound, a default value is assigned as the scaling factor.

17. A time-of-flight (ToF) ranging system, comprising: a ToF sensor configured to emit light pulses and generate a histogram based on reflected light pulses; as well as a processor coupled to the ToF sensor, wherein the processor is configured to: searching for a rising edge having a gradient in a first region of the histogram, wherein the gradient is greater than a threshold or the gradient is a maximum gradient in the first region, wherein the rising edge is in a first histogram bin having a first value; determining a second value for a second histogram bin in the first region, wherein the first histogram bin precedes the second histogram bin by a predetermined number; estimating a ratio between the first region of the histogram and a pre-stored light pulse shape based on the first value and the second value; scaling the pre-stored optical pulse shape by the estimated ratio; as well as A scaled pre-stored light pulse shape is subtracted from the histogram.

18. The ToF ranging system of claim 17, wherein the first region comprises a plurality of histogram bins around a reference zero histogram bin, wherein a distance of the reference zero histogram bin corresponds to a distance between a ToF transmitter of the ToF ranging system and a cover glass of the ToF ranging system.

19. The ToF ranging system of claim 18, wherein the pre-stored light pulse shape is a normalized histogram of light pulses emitted by the ToF sensor.

20. The ToF ranging system according to claim 19, wherein the processor is further configured to: After the subtraction, the histogram is used to estimate the distance of the target of the ToF ranging system.

Images