Time-of-flight rising edge ranging method with pulse distortion immunity

Abstract

The invention relates to a time-of-flight rising edge ranging method with pulse distortion immunity. A method of operating a time of flight (ToF) ranging system includes transmitting, by a transmitter, an optical signal toward one or more targets; receiving, by the ToF sensor, an optical signal reflected by the one or more targets; generating a histogram based on the received optical signal; estimating a gradient of a histogram box of the histogram by calculating a difference between adjacent histogram boxes; identifying one or more pulse regions in the histogram; finding a rising edge of which the gradient is greater than a preset threshold value or is the maximum gradient in the pulse region in the pulse region, wherein the rising edge is the leftmost rising edge in the pulse region with the gradient; finely adjusting the position of the rising edge; and calculating an estimate of the distance of the target in the pulse region by adding a predetermined offset to the distance of the rising edge.

Description

Technical Field

[0001] The present invention generally relates to time-of-flight (ToF) ranging systems, and in particular embodiments, to ToF ranging systems capable of measuring the distance to a target and having immunity to pulse shape distortion. Background Art

[0002] Time-of-flight (ToF) imagers (also called ToF sensors) have recently been widely used in various applications such as gesture / facial recognition, light detection and ranging (LiDAR), virtual reality, augmented reality, and autonomous robotics. ToF ranging systems use ToF imagers to measure the distance of an object (e.g., a target). To measure an object, a ToF ranging system instructs the ToF sensor to send a light signal (e.g., a light pulse) to the object and measures the time it takes for the signal to propagate to the object and back.

[0003] A single photon avalanche diode (SPAD) can be used as a detector of reflected light. In some applications, a SPAD array is provided as a sensor (e.g., a SPAD array) to detect reflected light pulses. The reflected photons can generate carriers in the SPAD by the photoelectric effect. The photon-generated carriers can trigger an avalanche current in one or more SPADs in the SPAD array. The avalanche current can signal an event, i.e., a photon has been detected. Information related to the reflection intensity (also referred to as a "signal count") is output as a histogram of the SPAD array. The histogram of each SPAD includes a plurality of histogram bins, wherein each histogram bin corresponds to a distance (or a narrow range of distances) from the SPAD array, and the value of each histogram bin (e.g., a signal count) corresponds to the number of avalanche current events detected (e.g., the number of photons detected).

[0004] The histogram from the SPAD needs to be processed to extract useful information, such as the number of detected targets, the distance of the targets, etc. The processing of the histogram remains challenging. For example, when the shape of the received light pulse is distorted and no longer resembles the shape of the emitted light pulse (e.g., emitted by a ToF sensor), and / or when the reflected light pulses from multiple targets merge together, it may be difficult to detect the distance of (multiple) targets. In addition, the processing of the histogram is typically computationally intensive and is often performed by an off-chip processor located in a semiconductor chip different from the semiconductor chip with the SPAD array. Efficient processing of the histogram of the SPAD is required to reduce the computational complexity and achieve a higher level of integration of ToF imagers and / or ToF ranging systems. Summary of the invention

[0005] In an embodiment, a method for ranging using a time-of-flight (ToF) ranging system includes: generating a histogram by a ToF imager of the ToF ranging system; identifying one or more pulse regions in the histogram, wherein each of the one or more pulse regions corresponds to a reflected light signal from at least one target of the ToF ranging system; generating a differential histogram by calculating differences between adjacent histogram bins in the histogram, wherein the differential histogram indicates gradients of the histogram bins; finding a first rising edge having a first gradient indicated by the differential histogram in a first pulse region of the one or more pulse regions, wherein the first gradient is greater than a threshold or is a maximum gradient in the first pulse region; fine-tuning a position of the first rising edge; and after fine-tuning the position of the first rising edge, calculating an estimate of the distance of a first target in the first pulse region by adding a predetermined offset to the distance of the first rising edge.

[0006] In an embodiment, a method of operating a time-of-flight (ToF) ranging system includes: emitting a light signal by a transmitter of a ToF sensor of the ToF ranging system toward one or more targets; receiving a light signal reflected by the one or more targets by the ToF sensor; generating a histogram based on the received light signal; estimating a gradient of a histogram bin of the histogram by calculating the difference between adjacent histogram bins; identifying one or more pulse regions in the histogram; finding a rising edge in a pulse region of the one or more pulse regions whose gradient is greater than a predetermined threshold or is the maximum gradient in the pulse region, wherein the rising edge is the leftmost rising edge in the pulse region having the gradient; fine-tuning the position of the rising edge; and calculating an estimate of the distance of the target in the pulse region by adding a predetermined offset to the distance of the rising edge.

[0007] In an embodiment, a time-of-flight (ToF) ranging system includes: a ToF sensor configured to receive reflected light pulses from one or more targets and generate a histogram based on the reflected light pulses; and a processor coupled to the ToF sensor, wherein the processor is configured to: identify one or more pulse regions in the histogram; find a rising edge in a pulse region of the one or more pulse regions whose gradient is greater than a predetermined threshold or is the maximum gradient in the pulse region; fine-tune the position of the rising edge by calculating an adjustment term for the position of the rising edge or by performing a fitting process between the rising edge and a pre-stored high-resolution rising edge, wherein the pre-stored high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of light pulses emitted by a light source of the ToF sensor; and calculate an estimate of the distance of a target in the pulse region by adding a predetermined offset to the distance of the rising edge after fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Details of one or more embodiments of the present invention are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of the present invention will be apparent from the description and drawings, and from the claims. In the drawings, the same or similar reference numerals generally represent the same or similar components in the various figures, and for the sake of brevity, these components will generally not be re-described. For a more complete understanding of the present invention, reference is now made to the following description in conjunction with the accompanying drawings, in which:

[0009] Figure 1 A time-of-flight (ToF) ranging system in an embodiment is illustrated;

[0010] Figure 2 illustrates a histogram generated by a ToF imager in an embodiment;

[0011] Figure 3 illustrates challenges with target range estimation due to limitations in receiver bandwidth in some embodiments;

[0012] Figure 4A-4C illustrates the challenges of target distance estimation in other embodiments;

[0013] Figure 5 illustrates the pile-up problem in a SPAD array in some embodiments;

[0014] Figure 6 FIGURES illustrate pulse regions in a histogram in an embodiment;

[0015] Figure 7 FIGURES illustrate pulse regions in a histogram in another embodiment;

[0016] Figure 8 A method of estimating a distance of a first target in a pulse region in an embodiment is illustrated;

[0017] Fig. 9 illustrates a histogram of pulses emitted by a ToF imager in an embodiment;

[0018] Fig.10 Graphs illustrate a histogram and a corresponding difference histogram in an embodiment;

[0019] Fig.11A and Fig. 11B together illustrate a method of performing a fitting process between a rising edge and a high-resolution rising edge in an embodiment;

[0020] Fig. 12A and Fig. 12B The diagram in the embodiment Figure 8 the execution of the method; and

[0021] Fig.13An integrated circuit (IC) device formed as a ToF ranging system in an embodiment is illustrated. DETAILED DESCRIPTION

[0022] The making and use of the currently preferred embodiments are discussed in detail below. However, it should be appreciated that the present invention provides many applicable inventive concepts that can be embodied in a variety of specific contexts. The specific embodiments discussed only illustrate the specific ways to make and use the present invention and do not limit the scope of the present invention.

[0023] The present invention will be described in the context of a time-of-flight (ToF) ranging system, and in a particular embodiment, a ToF ranging system capable of estimating the distance of a target with pulse distortion immunity will be described.

[0024] Figure 1 The time-of-flight (ToF) ranging system 100 in an embodiment is illustrated. Note that for simplicity, not all features of the ToF ranging system 100 are shown. In addition, for ease of discussion, Figure 1 An object 105 (eg, a target) and an ambient source 102 are illustrated, but it should be understood that the object 105 and the ambient source 102 are not part of the ToF ranging system 100 . Figure 1 The upper portion of FIG. 1 illustrates a cross-sectional view of the ToF ranging system 100 , while Figure 1 The lower portion of the diagram illustrates details of some components of the ToF ranging system 100, such as a top view of the SPAD array 101 and the reference SPAD array 101R, output pulses generated by the SPADs in the SPAD array 101 and the reference SPAD array 101R (e.g., indicating an avalanche current event), an OR tree used to combine the output pulses, and details of the processing unit 107.

[0025] like Figure 1 As illustrated in , the ToF ranging system 100 includes a SPAD array 101, a reference SPAD array 101R, a light source 103 (also referred to as a transmitter), and a processing unit 107 attached to a substrate 109 (eg, a printed circuit board (PCB), an interposer, etc.). Figure 1 An assembly housing 115 (e.g., a protective case, box, housing, etc.) is also illustrated. The SPAD array 101, the reference SPAD array 101R, the light source 103, the processing unit 107, and the substrate 109 are located in a space surrounded by the assembly housing 115. A cover glass 113 is attached to the upper surface of the side wall of the assembly housing 115, thereby forming a closed space to protect the components placed in the space. Figure 1 The cover glass 113 shown in FIG. 1 is attached to the assembly housing 115 by a non-limiting example only. The cover glass 113 may be attached by any suitable method.

[0026] Light source 103 is a circuit configured to generate a light signal 104 for illuminating an object 105 (e.g., a target). Light source 103 may be, for example, a vertical cavity surface emitting laser (VCSEL) device, a light emitting diode (LED), an infrared (IR) device, etc. Light signal 104 may include a plurality of light pulses. Reflected light signal 106 from object 105 is received by SPAD array 101. SPAD array 101 includes a plurality of SPADs arranged, for example, in rows and columns. Figure 1 In the example, the reference SPAD array 101R receives a reference light signal 104R from the light source 103 and provides a reference SPAD output. The reference SPAD output and the SPAD output (e.g., the output of the SPAD array 101) are sent to a ToF histogram generation circuit 107A to generate a histogram (also called a ToF histogram). Methods for generating histograms using SPAD arrays and reference SPAD arrays are known in the art and are not discussed in detail here. For example, US20170115381 discloses circuits and methods for generating histograms using SPAD arrays and reference SPAD arrays. US20170115381 is incorporated herein by reference.

[0027] exist Figure 1 In the example of , the ToF histogram generation circuit 107A is part of the processing unit 107. The processing unit 107 also includes a processor 107B, which receives and processes the histogram. The processor 107B can be, for example, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), etc. The processor 107B processes the histogram to extract useful information, such as the number of (multiple) targets in the histogram, the distance of each target, etc. In an example embodiment, the processor 107B is configured to perform one or more of the methods disclosed below to use the histogram to estimate the distance of one or more targets. In some embodiments, the ToF histogram generation circuit 107A and the processor 107B are integrated in the same semiconductor die (e.g., as a single integrated circuit (IC) die). In other embodiments, the ToF histogram generation circuit 107A and the processor 107B are formed in different (e.g., separate) semiconductor dies. These and other changes are fully intended to be included in the scope of the present disclosure.

[0028] Figure 1 A crosstalk signal 106C is illustrated, which represents the optical signal reflected by the cover glass 113 and received by the SPAD array 101. The detection and removal of the crosstalk signal from the histogram is known in the art and will not be discussed in detail here. The methods discussed herein assume that there is no or substantially no crosstalk signal in the histogram being processed for detecting the target, which can be achieved by removing the crosstalk signal from the histogram using methods known and used in the art.

[0029] In some embodiments, the components used to transmit light signals, receive reflected light signals, and generate histograms are collectively referred to as ToF imagers (also referred to as ToF sensors), and these components include light source 103, SPAD array 101, reference SPAD array 101R, and ToF histogram generation circuit 107A. Therefore, ToF ranging system 100 includes a ToF imager and a processor 107B, and processor 107B processes the histogram generated by the ToF imager.

[0030] Figure 1 The light source 103, SPAD array 101, reference SPAD array 101R and processing unit 107 are illustrated as separate components attached to substrate 109. This is merely a non-limiting example. In some embodiments, the light source 103, SPAD array 101, reference SPAD array 101R and ToF histogram generation circuit 107A are integrated into a single IC chip as an integrated ToF imager. In some embodiments, the light source 103, SPAD array 101, reference SPAD array 101R, ToF histogram generation circuit 107A and processor 107B are integrated into a single IC chip as an integrated ToF ranging system. These and other variations are fully intended to be included within the scope of the present disclosure.

[0031] Figure 2 A histogram generated by a ToF imager in an embodiment is illustrated. Figure 2 The histogram in the example has multiple boxes (also called histogram bins). The value of each box of the histogram (e.g., along the y-axis) represents the signal count within the duration of the box (e.g., a fixed duration). The position (e.g., along the x-axis) or index number of each box of the histogram corresponds to a distance (or a narrow range of distances) from the SPAD array 101. During histogram processing, when a target is identified as being located in a histogram bin, the distance of the histogram bin is used as the distance of the target (e.g., the distance between the SPAD array 101 and the target). In the discussion herein, it is assumed that the light source 103 and the SPAD array 101 are located at the same location, so that the distance between the SPAD array 101 and the target is considered to be the same as the distance between the light source 103 and the target.

[0032] For ease of discussion, each bin of the histogram is assigned a distance (e.g., the distance corresponding to the bin center), and the distance corresponding to the bin is also referred to as the bin distance or bin distance. The bin index (also referred to as bin index or bin number) can be used to indicate the bin distance. For example, assuming that the first bin of the histogram (e.g., the leftmost bin, or the earliest bin) has an index of 1, the second bin has an index of 2, and so on, the distance of the i-th bin can be calculated as ((i-1)+0.5)D, where D is the distance traveled by the light signal in a duration equal to half of the time interval T, and the time interval T is the duration of the histogram bin. The time interval T is also referred to as the bin time width, and the distance D is also referred to as the bin width D. Note that the time interval T represents the round-trip time of the light signal. In Figure 2 In the example of , the histogram includes two peaks 121 and 123, which may be caused by two targets at different distances from the ToF imager. Figure 2 The histogram in shows two pulse regions, with each of the peaks 121 and 123 located in one of the pulse regions. Figure 2 Each pulse region in the histogram corresponds to a reflected light signal from one target. In some embodiments, when two or more target positions are close, the reflected light signals from the two or more targets may merge together and form a (longer) pulse region.

[0033] Figure 3 The figure illustrates the challenges of target range estimation due to receiver bandwidth limitations in some embodiments. Figure 3 In FIG. 1 , the x-axis represents the range (e.g., distance) of the target, and the y-axis represents the range reported by the reference ToF ranging system. The reference ToF ranging system is a ToF ranging system that does not use the method disclosed herein and is used as a performance comparison with the ToF ranging system disclosed herein.

[0034] exist Figure 3 In the example shown, a reference ToF ranging system is determining the distance to a single target. Figure 3 A reference pulse shape 201 is illustrated in FIG. 2 , which represents the shape of a light pulse emitted by a ToF imager. Figure 3 Also illustrated are three histograms 211, 213, and 215, which are histograms of reflected light signals when the target is at a long distance (e.g., 1000 mm away), a short distance (e.g., 100 mm away), and a very short distance (e.g., 10 mm away), respectively. Figure 3In the figure, the histograms 211, 213 and 215 are illustrated as curves rather than bars, which are envelopes of the corresponding bar histograms, as is readily understood by those skilled in the art. In addition, for simplicity, in each of the histograms 211, 213 and 215, only the portion of the histogram containing the pulse corresponding to the reflection from the target (also referred to as the received pulse area or pulse area) is shown, while other portions of the histogram, such as portions containing random noise (e.g., shot noise), are drawn as flat lines (e.g., having zero values).

[0035] In order to determine the distance of the target, the received pulse area of ​​the histogram is identified with reference to the ranging system, the histogram box where the median position of the received pulse area is located is found, and the box distance of the histogram box found above is reported as the distance of the target. Here, the median position of the received pulse area refers to the position where the number of signal events (e.g., avalanche current events) before and after it is equal, and can be determined (e.g., approximated) as the midpoint between the starting position and the ending position of the received pulse area of ​​the histogram.

[0036] like Figure 3 As illustrated in , when the target is farther away, a small portion of the emitted light signal is reflected back, and the reflected photons from the target arrive at the ToF sensor at a lower rate. Therefore, the bandwidth of the SPAD array is not a limiting factor for detecting reflected photons. Therefore, the shape of the histogram 211 is substantially the same as the reference pulse shape 201. In other words, there is almost no distortion between the reference pulse shape 201 and the shape of the received pulse area of ​​the histogram 211. In this fairly ideal case, the distance to the median position of the received pulse area (indicated by the dotted line 212) gives an accurate estimate of the distance to the target, as indicated by the point on the dotted line 212. Note that this point is also on the dotted line 202 with a 45-degree angle, which represents an ideal output of Y=X.

[0037] When the target is closer, reflected photons arrive at a faster rate. The bandwidth of the SPAD array may limit the detection of photons. For example, if two photons arrive at the SPAD at two different times separated by a duration less than the SPAD's dead time, the first photon may be detected while the second photon may not be detected due to the SPAD's dead time after the first photon is detected. The effect of the bandwidth of the SPAD array will be discussed below. Figure 5 Discuss in more detail.

[0038] Due to the limited receiver bandwidth of the SPAD array, the shape of the received pulse area of ​​the histogram 213 is distorted compared to the reference pulse shape 201. In some embodiments, the received pulse area of ​​the histogram 213 has a shorter width than the ideal received pulse shape 223 (which is the same as the reference pulse shape 201), which is illustrated in dashed lines. The median position of the received pulse area of ​​the histogram 213 (as illustrated by dashed line 214) corresponds to a distance that is less than the median position of the ideal received pulse shape 223 (as illustrated by dashed line 214T). In other words, for short-range ranging, the reference ToF ranging system underestimates the distance of the target. The estimated distance generated by the reference ToF ranging system is indicated by a point on the dashed line 214T, which is below the 45-degree dashed line 202 representing the ideal (e.g., correct) distance estimate.

[0039] When the target is very close, the reflected photons arrive at an even faster rate. The bandwidth of the SPAD array may further limit the detection of reflected photons, resulting in greater distortion in the received pulse area of ​​the histogram 215. The corresponding ideal received pulse shape 225 is illustrated in dashed lines. The pulse area of ​​the histogram 215 and the median position of the ideal received pulse shape 225 are illustrated by dashed lines 216 and 216T, respectively. In this case, the reference ToF ranging system may seriously underestimate the target distance, which is shown as a point on the dashed line 216T. Please note that the vertical distance between the point on the dashed line 216T and the 45-degree angle dashed line 202 is greater than the vertical distance between the point on the dashed line 214T and the dashed line 202.

[0040] Note that although the bandwidth limitations of the SPAD array may distort the received pulse shape (e.g., the shape of the received pulse region of the histogram), the rising edge of the received pulse region of the histogram 213 (or 215) still overlaps (e.g., is identical to) the corresponding rising edge of the ideal received pulse shape. In other words, the position of the rising edge of the received pulse region (e.g., the bin index) is not affected by the distortion in the received pulse region. This important property observation is utilized in the method disclosed herein to estimate the distance to the target within the pulse region while being immune to pulse shape distortion. The method disclosed herein overcomes various challenges faced by reference ToF ranging systems. The details will be discussed below.

[0041] Figure 4A-4C Challenges in target distance estimation in other embodiments are illustrated. Figure 4A The figure shows a baseline scenario with a single target, where a ToF device (e.g., a ToF imager or a ToF ranging system) transmits a light signal toward the single target and generates a histogram based on the reflected light signal. The transmitted light pulse and received pulse areas of the histogram are Figure 4A, and are labeled "reference pulse" and "return pulse #1," respectively. The distance (e.g., in millimeters or meters) of the median position of return pulse #1 is proportional to the time difference Δ between the median position of the reference pulse and the median position of return pulse #1, and the distance of the median position of return pulse #1 gives an accurate estimate of the distance to a single target.

[0042] exist Figure 4B In the figure, the two targets labeled "target #1" and "target #2" are very close to each other. The reflected light pulses from the two targets are combined to form a combined received pulse area in the histogram, which is labeled "return pulse #1+#2". The distance of the median position of the combined received pulse area (the time difference Δ between the median position of the reference pulse and the median position of the combined received pulse area) is m Proportional to the distance of target #1 and target #2) can give an estimate of the distance between target #1 and target #2. This means that the reference ToF ranging system may not be able to distinguish between the two targets, and the estimated distance of the target is not an accurate estimate of the distance of either target. While algorithms can be developed to distinguish between the two targets contained in the histogram and estimate the distance of each target separately, such algorithms are generally complex and may be difficult to implement in hardware or software algorithms. However, in many applications, only the distance to the nearest target in the pulse region is of interest. For such applications, it may be advantageous to develop a simple algorithm to give an accurate estimate of the distance to the nearest target in the pulse region, such a simple algorithm can be easily implemented in hardware or software, and can reduce the power consumption of the implementation.

[0043] exist Figure 4C In this case, there is only one target, but the target is tilted so that different parts of the target are at different distances from the ToF device. As a result, the histogram may include regions of received pulses that are longer than the reference pulse shape (in Figure 4C The distance estimate based on the median position of return pulse #1 can give more or less the midpoint of the target. However, in many applications, only the closest point of the inclined target is of interest.

[0044] exist Figure 4B and Figure 4C Although the number of targets and the positions of the targets (multiple) distort the received pulse shape, Figure 4B The combined received pulse area (or Figure 4C The shape and position of the rising edge of the stretched received pulse region (in the stretched received pulse region) remain unchanged compared to the rising edge of the ideal received pulse shape. This observation is consistent with Figure 3As discussed below, based on these observations, the rising edge in the first pulse region of the histogram (e.g., the leftmost pulse region or the earliest arriving pulse region) is identified and used to obtain an estimate of the distance of the target closest to the ToF imager. The process can be repeated for each other pulse region of the histogram to obtain an estimate of the target in each other pulse region, where the target is the earliest arriving target among one or more targets contained in each other pulse region. Further details will be discussed below.

[0045] Figure 5 The diagram illustrates the pile-up problem in a SPAD array, in some embodiments. The pile-up occurs due to the dead time of the SPADs in the SPAD array and / or due to the structure of the SPAD array (e.g., the OR tree used therein). Figure 5 , two SPADs (labeled "SPAD#1" and "SPAD#2") in a SPAD array are illustrated. For SPAD#1, multiple reflected photons arrive at times 231, 233, 235, and 237, which are shown as arrows. SPAD#1 is sensitive (e.g., ready to detect photons) before time 231, as shown by the logic high state of the signal labeled "SPAD#1 sensitive". At time 231, a photon arrives and causes an avalanche current event, and SPAD#1 generates an output pulse 241 in the SPAD output signal labeled "SPAD#1 output". A skilled person will readily appreciate that after an avalanche current event occurs in the SPAD, the SPAD cannot detect photons for a period of time, which is referred to as the dead time of the SPAD. The dead time is represented by the logic low state of the SPAD#1 sensitive signal. Note that the photon arriving at time 233 falls within the dead time of SPAD#1 and is therefore not detected and no SPAD output is generated. The photon arriving at time 235 is outside the previous dead time of SPAD #1 and thus generates a SPAD output (e.g., a pulse in the SPAD #1 output signal). The photon arriving at time 237 is within the previous dead time of SPAD #1 and thus does not generate a SPAD output.

[0046] For SPAD #2, two photons arrive at times 251 and 253 separated by a duration longer than the dead time of SPAD #2, and therefore each photon generates an output pulse in the SPAD #2 output signal.

[0047] Figure 5An OR tree in the SPAD array is further illustrated, the OR tree comprising one or more logical OR gates that perform a logical OR operation on the outputs of all SPADs in the SPAD array. In some embodiments, the output of the OR tree is used as the output of the SPAD array. Note that the output pulse 241 of SPAD#1 overlaps in time with the output pulse 261 of SPAD#2, and as a result, the two output pulses 241 and 261 are merged together by the OR tree to form a merged pulse 271 that is longer than the output pulses 241 and 261. It is worth noting that the histogram readout typically detects the rising edge of the pulse, and therefore, only one event is recorded from this elongated pulse consisting of the two merged SPAD pulses.

[0048] As discussed above, the dead time of the SPADs and OR trees in the SPAD array may cause some photons to be missed (e.g., not detected or not counted) and may cause distortion of the output pulses of the SPADs. This is called the pile-up problem. The pile-up may distort the received pulse shape in the histogram and may pose challenges to target detection and / or target distance estimation for the ToF ranging system. The currently disclosed ToF ranging system 100 can overcome the pile-up problem and generate accurate estimates of the distances of, for example, the nearest target (e.g., closest to the ToF imager) and the first target within each pulse region by using the various methods disclosed herein.

[0049] Figure 6 The pulse area in the histogram in the embodiment is shown. Figure 6 In the example of , the ToF ranging device sends a light signal toward two targets labeled "target #1" and "target #2". The reflected light signals from target #1 and target #2 generate two pulse areas in the histogram, where the two pulse areas are labeled "return pulse #1" and "return pulse #2". For ease of discussion, the pulse area of ​​return pulse #1 is also referred to as pulse area 281, and the pulse area of ​​return pulse #2 is also referred to as pulse area 283. Figure 6 In the illustrated embodiment, the distance between the two targets is large enough so that the corresponding pulse regions 281 and 283 are separated from each other (eg, do not overlap). Figure 6 Also illustrated is a reference pulse 280, which corresponds to the shape of a light pulse emitted by a ToF ranging device. The shapes of pulse regions 281 and 283 may or may not be distorted compared to reference pulse 280. Figure 6In the example of FIG. 1 , pulse region 281 includes (e.g., corresponds to) one target (e.g., target #1), and pulse region 283 includes (e.g., corresponds to) one target (e.g., target #2). The shape and position of the rising edge of each of pulse regions 281 and 283 remain substantially unchanged compared to the shape and position of the rising edge of the corresponding ideal received pulse shape (e.g., the ideal received light pulse reflected by target #1 or target #2).

[0050] Figure 7 The pulse area in a histogram in another embodiment is illustrated. Figure 7 The examples in Figure 6 The example in is similar, but Figure 7 In the example shown in Figure 1, there are three targets, labeled "Target #1", "Target #2", and "Target #3". Target #2 and Target #3 are located close to each other, so the reflected signals from these two targets combine to form Figure 7 The pulse region 284 in is longer than the reference pulse 280 and is severely distorted compared to the reference pulse 280. Figure 7 In the example of FIG. 2 , pulse region 281 includes (e.g., corresponds to) one target (e.g., target #1), and pulse region 284 includes (e.g., corresponds to) a cluster of targets (e.g., target #2 and target #3). The shape and position of the rising edge of pulse region 281 are substantially unchanged from the shape and position of the rising edge of the ideal received pulse shape reflected from target #1. The shape and position of the rising edge of pulse region 284 are substantially the same as the shape and position of the rising edge of the ideal received pulse shape reflected from target #2, which is the first target (e.g., the earliest target or the nearest target) among the cluster of targets in pulse region 284.

[0051] Various embodiment methods are discussed below that can generate an estimate of the distance of the first target (e.g., the nearest target) in each pulse region in the histogram. The disclosed method can be applied to each pulse region of the histogram to generate an estimate. For example, the disclosed method can process pulse region 281 to generate an estimate of the distance of the first target (e.g., target #1) in pulse region 281. Note that since pulse region 281 is the earliest arriving pulse region in the histogram, the first target in pulse region 281 is also the target closest to the ToF imager (also referred to as the global nearest target). As another example, the disclosed method can process pulse region 284 to generate an estimate of the distance of the first target (e.g., target #2) in pulse region 284 (the region contains a target cluster, where target #2 is the first target in the target cluster). Details will be discussed below.

[0052] Figure 8The method 350 of estimating the distance of the first target in the pulse area in an embodiment is illustrated. It should be understood that Figure 8 The embodiment methods shown in are merely examples of many possible embodiment methods. Those skilled in the art will recognize many variations, substitutions, and modifications. For example, additions, deletions, replacements, rearrangements, or repetitions may be made. Figure 8 The steps are shown in the figure.

[0053] refer to Figure 8 At block 351, a histogram generated by a ToF imager (e.g., by ToF histogram generation circuit 107A) is received, e.g., by Figure 1 The processor 107B receives the data from the histogram. At block 352, one or more pulse regions in the histogram are identified. Each pulse region is defined by a start bin index and a corresponding stop bin index. For example, Figure 7 , pulse region 281 includes a histogram bin between a start bin index labeled “start #1” and a stop bin index labeled “stop #1”, and pulse region 284 includes a histogram bin between a start bin index labeled “start #2” and a stop bin index labeled “stop #2”. Methods for identifying pulse regions in a histogram are known and used in the art. For example, U.S. Patent No. 11120104B2, U.S. Patent Application No. 17 / 697784, and U.S. Patent Application No. 18 / 064412 each disclose a method (multiple) for identifying pulse regions in a ToF histogram. U.S. Patent No. 11120104B2, U.S. Patent Application No. 17 / 697784, and U.S. Patent Application No. 18 / 064412 are incorporated herein by reference. Any suitable method may be used to identify a pulse region in a histogram.

[0054] At block 353, a differential histogram is generated by calculating the difference between adjacent boxes of the histogram, wherein the differential histogram indicates the gradient at the histogram box. In some embodiments, a differential histogram is generated by calculating the difference between adjacent boxes of the histogram and assigning the calculated difference to the corresponding box of the differential histogram. For example, the value of the i-th box of the histogram is marked as H(i), and the value of the i-th box of the differential histogram is marked as DH(i), then DH(i) can be calculated as DH(i)=H(i)-H(i-1). In some embodiments, assuming that the index of the histogram box starts from 1, a zero value is assigned to DH(1) so that the differential histogram DH has the same length as the histogram H (e.g., the number of histogram boxes is the same), and there is a one-to-one correspondence between the histogram boxes of the histogram H and the histogram boxes (also referred to as boxes) of the differential histogram DH. The value DH(i) of the i-th bin of the difference histogram DH is an approximation of (e.g., proportional to) the gradient of the histogram at the i-th bin of the histogram H, and is therefore used as the gradient of the i-th bin H(i) of the histogram H in the discussion herein.

[0055] At block 354, a first rising edge is found in a first pulse region of the histogram. The first rising edge has a first gradient indicated by the differential histogram, wherein the first gradient is greater than a threshold value, or is a maximum gradient in the first pulse region. Two different embodiment methods for finding the location of the first rising edge are discussed below.

[0056] In an embodiment, the position of the first rising edge is found by finding the first histogram bin (e.g., the leftmost histogram bin) in the first pulse region that has the largest gradient in the first pulse region. The first pulse region can be any pulse region in the histogram (e.g., Figure 6 one of the pulse regions 281 and 283, or Figure 7 281 and 284 in the pulse region). The lower boundary and the upper boundary (e.g., the bin index number) of the first pulse region are labeled "Start_bin" and "Stop_bin", respectively, and the first rising edge is found by finding the minimum bin index k between Start_bin and Stop_bin, where the gradient of the histogram H at the bin index k (labeled as DH(k)) is the maximum value (e.g., the maximum gradient value) of the difference histogram DH between the bin indexes Start_bin and Stop_bin. In other words, the position of the first rising edge is indicated by the bin index k, where the gradient DH(k) is the largest among all the gradients of the histogram H at the bins between the bin indexes Start_bin and Stop_bin, and k is the minimum bin index between Start_bin and Stop_bin that has the gradient DH(k).

[0057] In another embodiment, the first rising edge of the first pulse region is found by finding a histogram bin having a gradient greater than a threshold TH indicated by the difference histogram DH, the threshold TH being determined by the ambient noise level and a user-defined confidence level (e.g., a user-defined scaling factor). For example, to find the position of the first rising edge, the values ​​of the bins of the difference histogram DH in the first pulse region (e.g., bin indexes between Start_bin and Stop_bin) are compared with the threshold TH, and the first bin (e.g., the leftmost bin) in the first pulse region of the histogram H having a gradient value greater than the threshold TH is identified. The bin index of the identified first bin is used to indicate the position (or equivalently, the distance) of the first rising edge in the first pulse region of the histogram.

[0058] As an example, the threshold TH may be determined as follows. The environmental event (eg, avalanche current event caused by an ambient light signal) of each histogram bin is labeled as σ 2 , and the user-defined confidence level is labeled C, then the threshold TH is calculated as Note that in the above calculations, the environmental events for each box in the differential histogram are considered to be twice the environmental noise events for each box of the histogram (because the subtraction operation used to calculate the differential histogram doubles the power of the environmental event level). Due to the shot noise characteristics of light, the noise in each differential histogram box is the square root of twice the environmental events for each box of the histogram. The environmental noise events for each histogram box can be measured in advance using a calibration process, or can be measured periodically during the operation of the ToF ranging system. The user-defined confidence level C can be adjusted according to the environmental noise level to achieve reliable detection of the first rising edge. For example, if the environmental noise level increases, the user-defined confidence level C can be increased, and vice versa, to avoid erroneous detection of the first rising edge and achieve the target detection rate of the first rising edge. In addition to the methods discussed above, other suitable methods for determining the threshold TH are also possible and are fully intended to be included within the scope of the present disclosure.

[0059] At block 355, by adjusting the term D ADJ Added to the position of the first rising edge to fine-tune the position of the first rising edge in the first pulse region. Used to calculate the adjustment term D ADJ Different methods are possible. The following discusses methods for determining the adjustment term D ADJ Two different implementation methods.

[0060] In an embodiment, the adjustment term D is calculated using the value of the bin of the difference histogram DH at the location of the first rising edge found in block 354. ADJ , and in some embodiments, the adjustment term D is calculated using the values ​​of the bins of the histogram H around the location of the first rising edgeADJ For example, assuming that the position of the first rising edge is determined from block 354 to be located in the mth bin of the histogram, then the adjustment term D ADJ It is calculated as DADJ = DH(m) / (H(m)+H(m+1)). The adjustment term D ADJ It is calculated by dividing DH(m) by the sum of H(m) and H(m+1). In other words, the adjustment term D ADJ The shape of the histogram around the first rising edge is taken into account and thus fine adjustment of a distance less than the bin width D can be provided, thereby improving the accuracy of the estimate of the position (eg, distance) of the first rising edge. The fine adjustment in block 355 is optional and is omitted in some embodiments.

[0061] In another embodiment, the position of the first rising edge is fine-tuned by performing a fitting process between the first rising edge and a pre-stored high-resolution rising edge. In a ToF imager, the histogram is quantized to limit the memory resources required to store the histogram and to reduce the complexity of the timing circuits used for quantization. Therefore, although in theory the position of the rising edge for a continuous-time signal can be found by, for example, finding the position with the maximum gradient, in practice the accuracy of the position is only as good as the timing resolution of the histogram. For example, for a histogram with a bin time width of 250ps, the discrete resolution is approximately R=37.5mm. In addition, the first rising edge found in the histogram may be masked or shifted by ambient noise, resulting in an uncertainty of approximately ±2R in the initial estimate of the position of the first rising edge. Refer to the following Fig. 9 , Fig.10 , Fig.11A and Fig. 11B The fitting process discussed improves the accuracy of the estimate of the position of the first rising edge, and the details will be discussed below.

[0062] Please note that although the following reference Fig. 9 , Fig.10 , Fig.11A and Fig. 11B The discussion is about a histogram and fitting the first rising edge of the histogram with a pre-stored high resolution rising edge, but a skilled person will readily appreciate that the processing can be readily applied to the pulse region of a histogram by treating the pulse region as a histogram, and thus, the process can be applied to fitting the first rising edge in one of the pulse regions of a histogram with a pre-stored high resolution rising edge.

[0063] Fig. 9 A histogram of light pulses emitted by a ToF imager in an embodiment is shown. Fig. 9The histogram includes four regions defined by vertical dashed lines A, B, C, D, and E. Specifically, the histogram includes a rising edge region, a peak region, a falling edge region, and a long tail region. For ease of discussion, the rising edge region, the peak region, the falling edge region, and the long tail region may also be referred to as the rising edge, the peak, the falling edge, and the long tail, respectively. As discussed above, pile-up can seriously distort the peak region, the falling edge region, and the long tail region. The long tail region may depend on the peak region, and the peak region may vary greatly in amplitude. Fig. 9 The various areas shown in the Fig.10 The example method illustrated in FIG. 1 is used to determine the

[0064] Fig.10 The histogram 301 and the corresponding difference histogram 303 (shown in dotted lines) in the embodiment are illustrated. As discussed above, the difference histogram 303 can be generated by calculating the difference between adjacent boxes of the histogram 301. The maximum and minimum values ​​of the difference histogram 303 are respectively marked as DH_MaxVal and DH_MinVal. In an embodiment, the first histogram box with the maximum value DH_MaxVal in the histogram 301 (e.g., the leftmost histogram box at position 304) is regarded as the starting point of the first rising edge region and is used as a rough estimate of the position of the first rising edge in the histogram 301. The index of the first histogram box with the maximum gradient value DH_MaxVal in the histogram is marked as start_ind, and the rising edge region includes a total of M histogram boxes in the histogram, starting from the box index start_ind. The number M can be, for example, 4, 5, 6, etc., depending on the transmission pulse width and the box time width. As mentioned previously, the starting point of the first rising edge region (eg, bin index start_ind) may also be determined by finding the first histogram bin (eg, the leftmost histogram bin) in the histogram having a gradient value greater than a predetermined threshold.

[0065] Fig.10 The histogram 301 in has a maximum value at position 306, and the difference histogram 303 at position 306 has a zero value (or close to a zero value due to quantization in the histogram). At position 308, the difference histogram has a minimum value DH_MinVal, which indicates the starting point of the falling edge region. The peak region is located between position 308 and the end of the rising edge region. At position 310, the difference histogram has the first zero value (or close to a zero value) after position 308. Position 310 indicates the end of the falling edge region and the beginning of the long tail region.

[0066] As discussed above, the index start_ind indicates the position (e.g., starting point) of the rising edge. Since the histogram generated in the ToF ranging system usually has a limited timing resolution, in order to reduce the memory requirements and the complexity of the timing circuit used to generate the histogram, the index start_ind only provides a rough estimate of the position of the rising edge. The rough estimate can be improved by a fitting process of fitting the rising edge with a pre-stored high-resolution rising edge, the details of which are discussed below.

[0067] In some embodiments, the pre-stored high-resolution rising edge is a portion (e.g., a beginning portion) of a pre-stored high-resolution histogram. The pre-stored high-resolution histogram is a histogram of transmit pulses of the ToF transmitter 103 and is generated, for example, by a calibration process, wherein the transmit pulses from the ToF transmitter 103 (see, e.g., Figure 3 The reference pulse shape 201 in FIG. 2 is reflected by a single target with little or no ambient noise. In other words, the pre-stored high-resolution histogram represents Figure 3 The ideal histogram of the reference pulse shape 201 in the ToF ranging system 100 is a histogram, and may be very similar to the reference pulse shape 201. It is noteworthy that the pre-stored high-resolution histogram is generated at a higher sampling frequency than the histogram generated during normal operation of the ToF ranging system 100, so that a higher timing resolution is achieved. After being generated, the pre-stored high-resolution histogram is stored in, for example, a memory area of ​​the ToF ranging system 100 for later use. For ease of discussion, the pre-stored high-resolution histogram may also be referred to as a high-resolution histogram. In some embodiments, when only the rising edge of the high-resolution histogram is used in the fitting process, the beginning part of the high-resolution histogram (e.g., the part containing the rising edge) is stored, and other parts are not stored to save memory space. Note that the high-resolution histogram can be analyzed and processed before the ToF ranging system 100 uses the high-resolution histogram so that the boundaries (e.g., bin indexes) of each area of ​​the high-resolution histogram (e.g., rising edge area, falling edge area, peak area) are determined and used later (e.g., for the fitting process).

[0068] In the illustrated embodiment, the timing resolution of the high-resolution histogram is N times the timing resolution of the histogram generated by the ToF ranging system 100. In other words, each histogram bin of the histogram of the ToF ranging system 100 corresponds to N adjacent histogram bins of the high-resolution histogram, and thus, each of the N adjacent histogram bins of the high-resolution histogram corresponds to one of the N sub-phases (e.g., time phases) of the histogram bin of the ToF ranging system 100. For example, if the rising edge of the histogram has M histogram bins, then the corresponding high-resolution rising edge has M×N histogram bins. The number N can be any suitable number, such as 4, 8, 10, 20, or more.

[0069] In some embodiments, a high-level description of the fitting process between the first rising edge and the high-resolution rising edge can be described as follows: the high-resolution rising edge is downsampled N times at each of the N substages to generate N coarse rising edges, where each of the N coarse rising edges corresponds to one of the N substages. In other words, the high-resolution rising edge is downsampled N times, each time starting from a different bin index (e.g., 1, 2, ..., N) to generate N coarse rising edges. Considering the example of N=4, downsampling the high-resolution rising edge starting from bin index 1 generates a coarse rising edge that includes bins 1, 5, 9, ..., etc. of the high-resolution rising edge; and downsampling the high-resolution rising edge starting from bin index 2 generates another coarse rising edge that includes bins 2, 6, 10, ..., etc. of the high-resolution rising edge. In the above N=4 example, each histogram bin has four substages, and each of the four coarse rising edges generated by downsampling corresponds to one of the four substages.

[0070] Next, an error metric is calculated for each coarse rising edge. The first rising edge and the nth coarse rising edge are labeled as rising edge (RisingEdge) and rising edge coarse_n (RisingEdgeCoarse_n), respectively, where n = 1, 2, ..., N, then the error metric between the first rising edge and the nth coarse rising edge (labeled as ErrorMetric(n)) can be calculated as:

[0071]

[0072] where M is the number of histogram bins in the first rising edge region and the function abs() gives the absolute value (eg amplitude) of the input variable.

[0073] Next, find the minimum value of the error metric ErrorMetric(n) and use the index of the minimum error metric to provide an adjustment term D for a rough estimate of the location of the first rising edge. ADJ (For example, D ADJ= k / N). For example, if ErrorMetric(k) has a minimum value and the bin index start_ind of the histogram is a rough estimate of the position of the first rising edge, then the updated (e.g., adjusted or fine-tuned) position of the first rising edge is given by start_ind + k / N. Thus, the fitting process provides a fractional adjustment to the rough estimate to improve the accuracy of the estimated position of the first rising edge.

[0074] The above fitting process illustrates high-level details. For actual implementation, more consideration needs to be given to the fitting process. For example, the starting point of the first rising edge determined from the histogram and the starting point of the high-resolution rising edge may not be completely aligned (e.g., may deviate from one or more histogram bins on either side). Therefore, the fitting process may need to search for the best starting bin index (which corresponds to the best substage that gives the minimum error metric) so that the high-resolution rising edge is downsampled around the initial bin index in both directions. As another example, the amplitude of the reflected pulse shape may change due to channel conditions, and therefore in order to correctly calculate the error metric, a normalization process may be performed on the first rising edge and the coarse rising edge. This can be achieved by, for example, scaling the first rising edge and the coarse rising edge so that at a particular bin position (e.g., the fourth bin of the rising edge and the fourth bin of the coarse rising edge), the first rising edge and the coarse rising edge have the same value (e.g., unit value 1) after normalization. As another example, since the search around the initial bin index is performed to find the best starting bin index for downsampling, downsampling may be performed more than N times. Refer to the following Fig.11A and Fig. 11B Discuss an example method for performing the fitting process.

[0075] Fig.11A and Fig. 11B 4 and 5 illustrate a method 400 for performing a fitting process between a rising edge (eg, a first rising edge) of a histogram and a pre-stored high-resolution rising edge in an embodiment. It should be understood that Fig.11A and Fig. 11B The embodiment methods shown in are merely examples of many possible embodiment methods. Those skilled in the art will recognize many variations, substitutions, and modifications. For example, additions, deletions, replacements, rearrangements, or repetitions may be made. Fig.11A and Fig. 11B The steps are shown in the figure.

[0076] refer to Fig.11A and Fig. 11B, at block 401, an initialization process is performed. For example, the iteration index iteration is initialized to a value of 1. The value Nval indicates the sub-stage during the fitting process to achieve the minimum error metric, which is initialized to 1. The value sign indicates the search direction for the best starting bin index for downsampling, which is initialized to a value of 1, for example. The value step_size indicates the step size of the starting bin index change during the search for the best starting bin index, which is initialized to 1. The value oscillation_count tracks the number of oscillations of the error metric value during the search for the best starting bin index, which is initialized to 0.

[0077] At block 407, a rising edge (e.g., a first rising edge) of the histogram is determined, for example, by finding a bin index start_ind. The details have been discussed above and are therefore not repeated. The rising edge region includes a total of M histogram bins in the histogram, starting from the bin index start_ind. The starting point of the rising edge region can be found by finding the first histogram bin with the maximum gradient value, or by finding the first histogram bin with a gradient value greater than a predetermined threshold, as discussed above. The rising edge is then normalized at block 409. The normalization process is as discussed above and is not described in detail.

[0078] At block 403, starting from bin index Nval, the pre-stored high-resolution rising edge is downsampled N times to generate a rough rising edge. At block 405, the rough rising edge is normalized. The normalization process is as discussed above and will not be described in detail.

[0079] At block 411, an error metric between the normalized coarse rising edge and the normalized rising edge is calculated using equation (1). Note that during calculation, the normalized coarse rising edge and the normalized rising edge correspond to the coarse rising edge and the rising edge in equation (1), respectively.

[0080] At block 413, the calculated error metric is compared to the error metric calculated during the previous iteration (referred to as the previous error metric). The previous error metric may be initialized (e.g., in block 401) with a large value (e.g., 100 or 1000) that is greater than the expected maximum value for the error metric. If the calculated error metric is greater than the previous error metric, this means that the search for the best starting bin index is in the wrong direction. Therefore, processing proceeds to block 415, where the value sign is reversed (e.g., changed from 1 to -1, or vice versa) so that the search for the best starting bin index for downsampling will continue in the opposite direction of the current search direction. After processing at block 415, processing continues to block 417. However, if the calculated error metric is less than the previous error metric, processing goes directly from block 413 to block 417.

[0081] At block 417, the step size step_size is adjusted to speed up the search process. In some embodiments, the calculated error metric is compared to a threshold. If the calculated error metric is above the threshold, this may indicate that the current starting index bin is far from the optimal starting index bin, so the value step_size is scaled by a value A, where A is an integer value greater than one, such as 2. If the calculated error metric is below the threshold, the step size does not need to be adjusted. In some embodiments, the processing in block 417 is omitted.

[0082] Next, at block 419, the calculated error metric is compared to a threshold value Δ, which has a small value (e.g., 0.1, 0.01, 0.001, or less). If the calculated error metric is less than Δ, the current starting bin index is considered to be close enough to the best starting bin index, and the process (e.g., iteration) of searching for the best bin index is stopped. If the calculated error metric is greater than Δ, processing continues to block 421.

[0083] At block 421, the method checks whether the search process (e.g., search for the best bin index) oscillates around the same position. If the current error metric calculated during the i-th iteration is equal to the error metric calculated during the (i-2)-th iteration, oscillation occurs. If no oscillation occurs during the current iteration, the process goes to block 427. If oscillation occurs, the process goes to block 423. At block 423, the oscillation_count value is increased by 1, and then the process goes to block 425. At block 425, the oscillation_count value is compared with a threshold value for the maximum number of oscillations, and if the oscillation_count value is greater than the maximum number of oscillations allowed, the search is stopped, otherwise the process goes to block 427.

[0084] At block 427, the iteration index iteration is compared with the threshold of the maximum number of iterations, and if the iteration index iteration is greater than the maximum number of iterations allowed, the search is stopped; otherwise, the process goes to block 429. At block 429, the value Nval is updated by adding the step size step_size (which can be a positive integer value or a negative integer value), and the iteration index iteration is increased by 1. Next, the process returns to block 403 for another iteration.

[0085] exist Fig.11A and Fig. 11B After the fitting process described in is completed, the value Nval represents the optimal starting bin index for downsampling the pre-stored high-resolution rising edge and is used to fine-tune the position of the first rising edge (indicated by start_ind) as follows: start_ind=start_ind+Nval / N.

[0086] Although Fig.11A and Fig. 11B The fitting process in the example is described with a rising edge as an example, but those skilled in the art will readily appreciate that the principles of the fitting process can be readily adapted to fit a falling edge of a histogram with a pre-stored high-resolution falling edge, fit a peak region of a histogram with a pre-stored high-resolution peak region, or fit another region of a histogram having a unique shape with a corresponding pre-stored high-resolution shape. These and other variations are fully intended to be included within the scope of the present disclosure.

[0087] Return to reference Figure 8 , after fine-tuning the position of the first rising edge, the predetermined offset D OFF The distance of the first rising edge in the first pulse region is added to form an estimate of the distance of the first target in the first pulse region. In some embodiments, the predetermined offset D OFF is half the width of the transmitted light pulse (measured in units of distance). For example, let T be the duration of the transmitted light pulse. P (See, for example, Figure 3 The duration T of the reference pulse shape 201 in P ), let C be the speed of light, then D OFF =0.5×T P ×C. Fig.10 , position 305 shows the estimated position of the first target, which is delayed by D from the first rising edge at position 304. OFF .

[0088] Please note that the predetermined offset D OFF The predetermined offset D is determined by the shape of the emitted light pulse generated by the light source 103 (eg, a VCSEL device), which shape may be pre-characterized (eg, measured), for example, by a calibration process of the light source 103. OFF is predetermined, saved, and later used to calculate the distance to the target. Note that due to the predetermined offset D OFF It is independent of various factors that distort the shape of the received pulse region in the histogram, so the estimate of the distance to the nearest target is robust and immune to distortions in the received pulse shape.

[0089] In some embodiments, the shape of the emitted light pulse generated by the light source 103 varies with the temperature of the light source 103. To improve the accuracy of the estimate of the distance to the nearest target, the shape of the emitted light pulse may be characterized (e.g., measured) at multiple temperatures (e.g., covering all or part of the operating temperature range of the light source 103), and predetermined offsets D at different temperatures may be saved, for example, in a lookup table (LUT). OFFDuring operation of the ToF ranging system, the current temperature of the light source 103 is determined, and a predetermined offset D may be used. OFF The stored value determines the predetermined offset D at the current temperature OFF For example, a simple LUT method finds the temperature in the LUT that is closest to the current temperature and uses the corresponding stored predetermined offset D OFF As the predetermined offset D at the current temperature OFF As another example, a predetermined offset D at different temperatures may be used. OFF The stored value is interpolated (e.g., linear interpolation) to generate a predetermined offset D at the current temperature. OFF interpolation.

[0090] Fig. 12A and Fig. 12B The diagram in the embodiment Figure 8 The execution of the method. Fig. 12A In FIG. 5 , the x-axis represents the target distance and the y-axis represents the distance reported by the ToF ranging system. Curve 501 shows Figure 8 For comparison, curve 503 shows the ranging result of the reference ranging system, which estimates the distance of the target based on the median position of the received pulse area. In addition, Fig. 12A The dotted line with an angle of 45 degrees in FIG. 1 shows an ideal distance measurement result (Y=X). Fig. 12B The range error at different target distances is shown. Fig. 12A and Fig. 12B In the simulation of , the reflected photons from the target arrive at the ToF sensor at a high rate, so the pile-up problem occurs. Curve 501 corresponds to a ranging method in which the first rising edge is found by finding the first bin with the maximum gradient value in the pulse area, and the adjustment term for fine-tuning is calculated using the gradient value and the value of the histogram bin.

[0091] from Fig. 12A and Fig. 12B It can be seen that the disclosed method generates a good estimate of the target distance over a wide range of target distances, while the reference method underestimates the target distance at close distances. The maximum ranging error of the disclosed method is approximately ±5 mm, while the maximum ranging error of the reference method is approximately -33 mm.

[0092] Fig.13 An integrated circuit (IC) device 1000 is shown which is formed as a ToF ranging system in an embodiment. The IC device 1000 may be formed as Figure 1ToF ranging system 100. Note that for simplicity, not all features of IC device 1000 are shown. IC device 1000 includes a light source 1013 (eg, VCSEL, LED, etc.) and a SPAD array 1010. Light source 1013 may correspond to Figure 1 The SPAD array 1010 may correspond to Figure 1 The SPAD array 101 and the reference SPAD array 101R may further include Figure 1 ToF histogram generating circuit 107A.

[0093] IC device 1000 further includes a memory module 1015 for storing a histogram generated by SPAD array 1010. It is worth noting that in FIG. 11 , processor 1017 is integrated in IC device 1000, and processor 1017 may correspond to Figure 1 The processor 107B in the IC device 1000 may be configured to perform one or more methods disclosed herein to generate an estimate of the distance of a target in the pulse region. The IC device 1000 may also include an input / output (I / O) module 1019 for communicating with other devices.

[0094] Note that the various methods disclosed for target distance estimation are easy to implement and require very little computing power. These advantages allow many different types of processors to be easily and economically integrated into the IC device 1000 as the processor 1017, and no other processor is required to perform off-chip processing of the histogram. The on-chip processing of the histogram data significantly reduces the chip complexity associated with transferring the histogram data I / O to an off-chip processing module and reduces the processing delays associated with off-chip processing.

[0095] Variations and modifications to the disclosed method are possible and are fully intended to be included within the scope of the present disclosure. For example, the falling edge of the pulse region in the histogram corresponds to the reflection from the last target (e.g., the farthest target) in the target cluster in the pulse region, and is also highly robust to various factors that distort the return pulse shape. Therefore, the falling edge of the pulse region can be used to determine the position of the last target in the target cluster in the pulse region. For example, once the falling edge of the pulse region in the histogram is determined, the distance to the falling edge can be determined by subtracting a predetermined offset D from the distance to the falling edge. OFF to estimate the position of the last target, where the distance of the falling edge is the distance of the box containing the starting point of the falling edge, such as Fig. 9 As shown by the dashed line C in FIG. In addition, the position of the falling edge can be fine-tuned by performing a fitting process between the falling edge and the pre-stored high-resolution falling edge. Once a skilled technician has read the disclosure of this article, it should be easy to adjust the disclosed fitting process to fit the falling edge.

[0096] The disclosed embodiments can achieve advantages. For example, various disclosed methods provide accurate estimates of the first target (e.g., the nearest target) in each pulse region of the histogram and are robust against distortions in the received pulse shape. Temperature compensation can be performed to further improve the accuracy of the estimate. Adjustments (e.g., provided by a fitting process or by calculation using a histogram and a differential histogram) provide fine adjustments of distances less than the bin width. The disclosed methods are easy to implement and save computing power, thus allowing a ToF ranging system to be fully integrated on a single IC device.

[0097] Example embodiments of the invention are summarized here. Other embodiments may also be understood from the entire specification and claims submitted herein.

[0098] Example 1. In an embodiment, a method for ranging using a time-of-flight (ToF) ranging system includes: generating a histogram by a ToF imager of the ToF ranging system; identifying one or more pulse regions in the histogram, wherein each of the one or more pulse regions corresponds to a reflected light signal from at least one target of the ToF ranging system; generating a differential histogram by calculating the difference between adjacent histogram bins of the histogram, wherein the differential histogram indicates a gradient of the histogram bin; finding a first rising edge having a first gradient indicated by the differential histogram in a first pulse region of the one or more pulse regions, wherein the first gradient is greater than a threshold or is a maximum gradient in the first pulse region; fine-tuning the position of the first rising edge; and after fine-tuning the position of the first rising edge, calculating an estimate of the distance of the first target in the first pulse region by adding a predetermined offset to the distance of the first rising edge.

[0099] Example 2, the method of Example 1, wherein the first rising edge is a leftmost rising edge in a first pulse region of the histogram having a first gradient.

[0100] Example 3, the method of Example 2, wherein the first pulse region corresponds to a reflected light signal from one or more targets of the ToF ranging system, wherein the first target is a closest target among the one or more targets.

[0101] Example 4, the method of Example 1, further includes: finding a second rising edge having a second gradient indicated by a differential histogram in a second pulse region of one or more pulse regions, wherein the second gradient is greater than a threshold or is a maximum gradient in the second pulse region; fine-tuning the position of the second rising edge; and after fine-tuning the position of the second rising edge, calculating an estimate of the distance of a second target in the second pulse region by adding a predetermined offset to the distance of the second rising edge.

[0102] Example 5, the method of Example 1, wherein the threshold is determined by the ambient noise level in the histogram and a user-specified scaling factor, wherein the predetermined offset is determined by the shape of the light pulse emitted by the ToF imager and corresponds to half the width of the light pulse.

[0103] Example 6, the method of Example 1, wherein the first rising edge is located in a histogram bin with bin index M, wherein fine-tuning the position of the first rising edge includes: calculating an adjustment term based on the first gradient and values ​​of the histogram bins at bin indexes M and M+1; and adding the adjustment term to the position of the first rising edge.

[0104] Example 7, the method of Example 1, wherein fine-tuning the position of the first rising edge includes performing a fitting process between the first rising edge and a pre-stored high-resolution rising edge.

[0105] Example 8, the method of Example 7, wherein the pre-stored high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of emitted light pulses from the ToF imager, wherein a timing resolution of the high-resolution histogram is N times the timing resolution of the histogram.

[0106] Example 9, the method of Example 8, wherein each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, the N adjacent histogram bins representing N substages of each histogram bin, wherein performing a fitting process includes: downsampling a pre-stored high-resolution rising edge N times at each of the N substages to generate N coarse rising edges; generating N error metrics by calculating the difference between the rising edge and the N coarse rising edges; and finding the substage having the lowest error metric among the N error metrics among the N substages.

[0107] Example 10, the method of Example 9, wherein the position of the rising edge is indicated by an index of a histogram bin of the histogram, wherein fine-tuning the position of the rising edge comprises adjusting the index to a fraction indicated by the substage having the lowest error metric.

[0108] Example 11, the method of Example 1, further comprising performing a calibration process for the ToF imager, comprising: measuring a shape of a light pulse emitted by the ToF imager at different temperatures; calculating a plurality of values ​​for a predetermined offset at different temperatures; and storing the plurality of values ​​for the predetermined offset at different temperatures.

[0109] Example 12, the method of Example 11, further comprising: obtaining a measurement of a current temperature of the ToF imager; and determining a current value of the predetermined offset based on the measurement of the current temperature and a plurality of stored values ​​for the predetermined offset at different temperatures.

[0110] Example 13. In an embodiment, a method of operating a time-of-flight (ToF) ranging system includes: emitting a light signal toward one or more targets by a transmitter of a ToF sensor of the ToF ranging system; receiving a light signal reflected by one or more targets by the ToF sensor; generating a histogram based on the received light signal; estimating a gradient of a histogram bin of the histogram by calculating the difference between adjacent histogram bins; identifying one or more pulse regions in the histogram; finding a rising edge in a pulse region of the one or more pulse regions whose gradient is greater than a predetermined threshold or is the maximum gradient in the pulse region, wherein the rising edge is the leftmost rising edge in the pulse region having the gradient; fine-tuning the position of the rising edge; and calculating an estimate of the distance of the target in the pulse region by adding a predetermined offset to the distance of the rising edge.

[0111] Example 14, the method of Example 13, wherein the predetermined threshold is determined by an ambient noise level in the histogram and a user-specified scaling factor, wherein the predetermined offset is proportional to half the width of a light pulse emitted by the emitter.

[0112] Example 15, the method of Example 13, wherein the rising edge is located in a histogram bin having a first bin index, wherein fine-tuning the position of the rising edge comprises: calculating an adjustment item based on a gradient of the rising edge, a first value of the histogram bin having the first bin index, and a second value of another histogram bin adjacent to the histogram bin; and adding the adjustment item to the first bin index.

[0113] Example 16, the method of Example 13, wherein fine-tuning the position of the rising edge includes performing a fitting process between the rising edge and a high-resolution rising edge, wherein the high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of light pulses emitted by the transmitter, wherein a timing resolution of the high-resolution histogram is N times the timing resolution of the histogram, so that each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, and the N adjacent histogram bins represent N substages of each histogram bin, wherein performing the fitting process includes: downsampling the high-resolution rising edge N times at each of the N substages to generate N coarse rising edges; generating N error metrics by calculating the difference between the rising edge and the N coarse rising edges; and finding a substage with a lowest error metric among the N error metrics among the N substages.

[0114] Example 17. The method of Example 16, wherein the rising edge is located in a histogram bin having a first bin index, wherein fine-tuning the position of the rising edge comprises adjusting the first bin index by an amount indicated by the substage having the lowest error metric.

[0115] Example 18. In an embodiment, a time-of-flight (ToF) ranging system includes: a ToF sensor configured to receive reflected light pulses from one or more targets and generate a histogram based on the reflected light pulses; and a processor coupled to the ToF sensor, wherein the processor is configured to: identify one or more pulse regions in the histogram; find a rising edge in a pulse region of the one or more pulse regions whose gradient has a gradient greater than a predetermined threshold or is the maximum gradient in the pulse region; fine-tune the position of the rising edge by calculating an adjustment term for the position of the rising edge, or by performing a fitting process between the rising edge and a pre-stored high-resolution rising edge, wherein the pre-stored high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of light pulses emitted by a light source of the ToF sensor; and calculate an estimate of the distance of the target in the pulse region by adding a predetermined offset to the distance of the rising edge after fine-tuning.

[0116] Example 19, a ToF ranging system according to Example 18, wherein the timing resolution of the high-resolution histogram is N times the timing resolution of the histogram, so that each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, and the N adjacent histogram bins represent N sub-stages of each histogram bin, wherein performing a fitting process includes: downsampling a pre-stored high-resolution rising edge N times at each of the N sub-stages to generate N coarse rising edges; generating N error metrics by calculating the difference between the rising edge in the histogram and the N coarse rising edges; and finding the sub-stage with the lowest error metric among the N error metrics among the N sub-stages.

[0117] Example 20, the ToF ranging system of Example 18, wherein the predetermined offset is proportional to half the width of the emitted light pulse.

[0118] Although the present invention has been described with reference to illustrative embodiments, this description is not intended to be interpreted in a limiting sense. Various modifications and combinations of the illustrative embodiments and other embodiments of the present invention will be apparent to those skilled in the art upon reference to this description. Therefore, the appended claims are intended to cover any such modifications or embodiments.

Claims

1. A method for measuring distance using a time-of-flight (ToF) ranging system, the method comprising: Generating a histogram by a ToF imager of the ToF ranging system; identifying one or more pulse regions in the histogram, wherein each of the one or more pulse regions corresponds to a reflected light signal from at least one target of the ToF ranging system; generating a difference histogram by calculating differences between adjacent histogram bins of the histogram, wherein the difference histogram indicates gradients of the histogram bins; Finding a first rising edge in a first pulse region of the one or more pulse regions, the first rising edge having a first gradient indicated by the difference histogram, wherein the first gradient is greater than a threshold value or the first gradient is a maximum gradient in the first pulse region; Fine-tuning the position of the first rising edge; as well as After fine-tuning the position of the first rising edge, an estimate of the distance of a first target in the first pulse region is calculated by adding a predetermined offset to the distance of the first rising edge. 2 . The method of claim 1 , wherein the first rising edge is a leftmost rising edge in the first pulse region of the histogram having the first gradient. 3 . The method according to claim 2 , wherein the first pulse region corresponds to a reflected light signal from one or more targets of the ToF ranging system, wherein the first target is a closest target among the one or more targets.

4. The method according to claim 1, further comprising: Finding a second rising edge in a second pulse region of the one or more pulse regions, the second rising edge having a second gradient indicated by the difference histogram, wherein the second gradient is greater than a threshold or the second gradient is a maximum gradient in the second pulse region; Fine-tuning the position of the second rising edge; as well as After fine-tuning the position of the second rising edge, an estimate of the distance of a second target in the second pulse region is calculated by adding a predetermined offset to the distance of the second rising edge.

5. The method of claim 1 , wherein the threshold is determined by an ambient noise level in the histogram and a user-specified scaling factor, wherein the predetermined offset is determined by a shape of a light pulse emitted by the ToF imager and corresponds to half the width of the light pulse.

6. The method of claim 1 , wherein the first rising edge is located in a histogram bin having a bin index M, wherein fine-tuning the position of the first rising edge comprises: calculating an adjustment term based on the first gradient and the values ​​of the histogram bins at bin indices M and M+1; as well as The adjustment item is added to the position of the first rising edge. 7 . The method of claim 1 , wherein fine-tuning the position of the first rising edge comprises performing a fitting process between the first rising edge and a pre-stored high-resolution rising edge.

8. The method of claim 7, wherein the pre-stored high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of emitted light pulses from the ToF imager, wherein a timing resolution of the high-resolution histogram is N times a timing resolution of the histogram.

9. The method of claim 8, wherein each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, the N adjacent histogram bins representing N sub-stages of each histogram bin, wherein performing the fitting process comprises: Downsampling the pre-stored high-resolution rising edge by a factor of N at each of the N sub-stages to generate N coarse rising edges; generating N error metrics by calculating differences between the rising edge and the N coarse rising edges; as well as A sub-stage having a lowest error metric among the N error metrics is found among the N sub-stages.

10. The method of claim 9, wherein the position of the rising edge is indicated by an index of a histogram bin of the histogram, wherein fine-tuning the position of the rising edge comprises adjusting the index by a fraction indicated by the substage having the lowest error metric.

11. The method of claim 1 , further comprising performing a calibration process for the ToF imager, comprising: measuring the shapes of light pulses emitted by the ToF imager at different temperatures; calculating a plurality of values ​​for the predetermined offset at the different temperatures; as well as The plurality of values ​​for the predetermined offset at the different temperatures are stored.

12. The method according to claim 11, further comprising: Obtaining a measurement value of a current temperature of the ToF imager; as well as A current value of the predetermined offset is determined based on the measured value of the current temperature and the plurality of stored values ​​for the predetermined offset at the different temperatures.

13. A method of operating a time-of-flight (ToF) ranging system, the method comprising: The ToF sensor of the ToF ranging system emits a light signal toward one or more targets; receiving, by the ToF sensor, the light signal reflected by the one or more targets; generating a histogram based on the received optical signal; estimating a gradient of a histogram bin of the histogram by computing the difference between adjacent histogram bins; identifying one or more pulse regions in the histogram; Find a rising edge in a pulse region among the one or more pulse regions, the gradient of which is greater than a predetermined threshold or the gradient is the maximum gradient in the pulse region, wherein the rising edge is the leftmost rising edge in the pulse region having the gradient; Fine-tuning the position of the rising edge; as well as An estimate of the distance of a target in the pulse region is calculated by adding a predetermined offset to the distance of the rising edge.

14. The method of claim 13, wherein the predetermined threshold is determined by an ambient noise level in the histogram and a user-specified scaling factor, and wherein the predetermined offset is proportional to half the width of a light pulse emitted by the emitter.

15. The method of claim 13, wherein the rising edge is located in a histogram bin having a first bin index, wherein fine-tuning the position of the rising edge comprises: calculating an adjustment term based on the gradient of the rising edge, a first value of the histogram bin having the first bin index, and a second value of another histogram bin adjacent to the histogram bin; as well as The adjustment term is added to the first bin index.

16. The method of claim 13, wherein fine-tuning the position of the rising edge comprises performing a fitting process between the rising edge and a high-resolution rising edge, wherein the high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of light pulses emitted by the emitter, wherein a timing resolution of the high-resolution histogram is N times a timing resolution of the histogram, such that each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, the N adjacent histogram bins representing N sub-stages of each histogram bin, wherein performing the fitting process comprises: Downsampling the high-resolution rising edge by a factor of N at each of the N sub-stages to generate N coarse rising edges; generating N error metrics by calculating differences between the rising edge and the N coarse rising edges; as well as A sub-stage having a lowest error metric among the N error metrics is found among the N sub-stages.

17. The method of claim 16, wherein the rising edge is located in a histogram bin having a first bin index, wherein fine-tuning the position of the rising edge comprises adjusting the first bin index by an amount indicated by the substage having the lowest error metric.

18. A time-of-flight (ToF) ranging system, comprising: a ToF sensor configured to receive reflected light pulses from one or more targets and generate a histogram based on the reflected light pulses; as well as a processor coupled to the ToF sensor, wherein the processor is configured to: identifying one or more pulse regions in the histogram; Find a rising edge in a pulse region of the one or more pulse regions, the gradient of which is greater than a predetermined threshold or the gradient is the maximum gradient in the pulse region; Fine-tuning the position of the rising edge by calculating an adjustment term for the position of the rising edge, or by performing a fitting process between the rising edge and a pre-stored high-resolution rising edge, wherein the pre-stored high-resolution rising edge corresponds to a rising edge of a high-resolution histogram of light pulses emitted by a light source of the ToF sensor; as well as An estimate of the distance of a target in the pulse region is calculated by adding a predetermined offset to the distance of the rising edge after the fine tuning.

19. The ToF ranging system according to claim 18, wherein the timing resolution of the high-resolution histogram is N times the timing resolution of the histogram, so that each histogram bin of the histogram corresponds to N adjacent histogram bins of the high-resolution histogram, and the N adjacent histogram bins represent N sub-stages of each histogram bin, wherein performing the fitting process comprises: Downsampling the pre-stored high-resolution rising edge by a factor of N at each of the N sub-stages to generate N coarse rising edges; generating N error metrics by calculating the difference between the rising edge in the histogram and the N coarse rising edges; as well as A sub-stage having a lowest error metric among the N error metrics is found among the N sub-stages.

20. The ToF ranging system of claim 18, wherein the predetermined offset is proportional to half the width of the emitted light pulse.

Images