Time-of-flight circuits and methods
The time-of-flight circuit and method address scaling challenges in dToF systems by using adaptive measurement cycles and confidence signals to reduce power consumption and improve reliability in high ambient light conditions, ensuring accurate distance determination.
Patent Information
- Application Number
- JP2024529492
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-26
- Filing Date
- 2022-11-21
- Publication Date
- 2024-11-15
AI Technical Summary
Existing direct time-of-flight (dToF) systems face challenges in scaling to higher resolutions and ambient light levels, leading to data congestion, high power dissipation, and reduced reliability due to dark count rates and ambient light interference.
A time-of-flight circuit and method that measures photon arrival times using test and reference time points within varying measurement intervals, incorporating a confidence signal to adaptively adjust measurement cycles and reduce power consumption by quenching SPADs during non-detection periods, allowing for efficient distance determination even in high ambient light conditions.
The solution enables reliable distance measurement with reduced power consumption and improved reliability by minimizing unnecessary measurements, effectively handling high ambient light levels and reducing dark count rates, thus enhancing the performance of dToF systems.
Smart Images

Figure 2024542469000001_ABST
Abstract
Description
[Technical field]
[0001] TECHNICAL FIELD This disclosure relates generally to time-of-flight circuits and methods for direct time-of-flight. [Background technology]
[0002] Generally, Time of Flight (ToF) devices are known, which can be distinguished between iToF (indirect ToF) and dToF (direct ToF), in which distance can be measured indirectly based on the phase shift of the emitted light compared to the detected light.
[0003] In dToF, distance can be measured directly by determining the round-trip delay of the emitted light. dToF can be based on diode technology such as SPAD (single photon avalanche diode).
[0004] In addition to being able to detect single photons, SPADs may have the inherent property that the arrival time of the photon is known. In several publications, SPADs have been proposed to be used for distance sensing, i.e. for LIDAR and 3D camera applications.
[0005] In dToF, a histogram can be generated showing the number of photons and their respective arrival times.
[0006] While the technology exists for dToF, it is generally desirable to provide ToF circuitry and methods. Summary of the Invention
[0007] According to a first aspect, the present disclosure provides a time-of-flight circuit for determining a round trip delay of photons emitted by a light source and incident on a time-of-flight imaging element, the time-of-flight circuit being configured to measure a test number of photons at a test time point and a reference number of photons at a reference time point and compare the test number of photons with the reference number of photons to determine the round trip delay of emitted photons, the test time point and the reference time point being included in one measurement time interval of at least two measurement time intervals during which the test time point is varied.
[0008] According to a second aspect, the present disclosure provides a time-of-flight method for determining the round trip delay of photons emitted by a light source and incident on a time-of-flight imaging element, the time-of-flight method comprising the steps of measuring a test number of photons at a test time point and a reference number of photons at a reference time point and comparing the test number of photons with the reference number of photons to determine the round trip delay of an emitted photon, the test time point and the reference time point being included in one measurement time interval of at least two measurement time intervals during which the test time point is varied.
[0009] Further aspects are set out in the dependent claims, the following description and the drawings. [Brief description of the drawings]
[0010] Embodiments will now be described, by way of example, with reference to the accompanying drawings, in which: [Figure 1] 1 shows a timing diagram according to the present disclosure. [Diagram 2] 1 illustrates a TOF receiver according to the present disclosure in which the reliability signal is constructed by an up / down counter. [Diagram 3] 1 illustrates a TOF receiver according to the present disclosure, where a confidence signal is constructed by a switch-cap averaging system. [Figure 4] A numerical statistical simulation based on the ToF receiver of FIG. 3 is shown. [Diagram 5] 1 illustrates a TOF method according to the present disclosure. [Figure 6]1 illustrates a flip-flop according to the present disclosure for determining object distance. [Figure 7] 1 is an embodiment of a time-of-flight method according to the present disclosure. [Figure 8] 10 is a further embodiment of a time-of-flight method according to the present disclosure, where the confidence values are further adapted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Before a detailed description of the embodiment is given, starting with FIG. 1, a general description will be given.
[0012] As mentioned at the beginning, direct time-of-flight (dToF) methods and devices are generally known. They can use SPAD (single photon avalanche diode) technology. However, it is recognized that SPADs can also be triggered by thermally generated minority carriers in semiconductors, or tunneling of carriers, also called dark count rate (DCR). Furthermore, when light is incident on a ToF receiver, the incident photons may correspond to emitted (or correlated) photons (ToF photons) resulting from reflections in the scene of a pulsed laser source, or to uncorrelated photons, such as from ambient light (AL) reflected on a region of interest in the scene. In known dToF systems, a constant level for the sum of AL and DCR is obtained by creating a histogram of arrival times, with the peaks correlating with the TOF arrival moments.
[0013] Creating many simultaneous histograms for array operations can be implemented for low numbers of pixels (e.g., linear arrays of pixels, or limited resolution arrays of pixels): the instant in time can be recorded using a counter to achieve time to digital conversion (TDC), and this data can be communicated to a digital signal processor (DSP) to achieve histogram construction, and finally a threshold to estimate distance. However, when scaling to higher resolutions and / or higher AL levels, data congestion and higher power dissipation can be difficult.
[0014] Thus, it may be desirable to provide small circuits that can be integrated on a per pixel basis, and it is recognized that they can be adapted to interpret incoming time-of-flight and ambient light photons.
[0015] It is recognized that this may be accomplished by successively assuming a range of distances as the object distance, during which each of these assumed distances is evaluated based on a confidence signal constructed from the incident photon rate occurring at that distance. If there is not enough confidence, the next distance can be considered. Many locations can be (rapidly) scanned such that within a reasonable time, an actual distance can be found that can be indicated by a high confidence signal.
[0016] Furthermore, it is recognized that power reduction can be achieved by lowering the number of measurement cycles per second while still maintaining state-of-the-art reliability levels.
[0017] For example, once confidence flags a change in distance, or during array operation, many pixels within the same illumination zone flag such a change, the number of measurement cycles per second can be rapidly increased to adapt to the reality of the changing scene and update to the new scene. By considering one assumed distance at a time, many incident photons can just be ignored so that a 20-100 times reduction in SPAD power dissipation can be achieved. By keeping the SPAD in a quenched position most of the time, dead time can become less relevant. Also, the effects of photon pile-up can be reduced in accordance with the present disclosure.
[0018] The in-pixel distance determination can further be complemented by data communication to a DSP with image data transfer rates comparable to standard 2D image sensors. Compared to other systems of in-pixel signal demodulation (e.g. Current Assisted Photonics Demodulators (CAPD)), it may not be necessary to work with different measurement frames, each of which must be recorded consecutively.
[0019] Thus, in accordance with the present disclosure, a 3D ToF image sensor can be provided, which can be fabricated with a single layer of electronics, but can also be implemented in a 3D-stacked configuration, where several semiconductor layers are stacked for different functions, such as a detection layer for the SPAD and a layer for subsequent data processing.
[0020] Thus, some embodiments relate to a time-of-flight circuit for determining the round trip delay of photons emitted by a light source and incident on a time-of-flight imaging element, the time-of-flight circuit being configured to measure a test number of photons at a test time point and a reference number of photons at a reference time point and compare the test number of photons with the reference number of photons to determine the round trip delay of an emitted photon, the test time point and the reference time point being included in one measurement time interval of at least two measurement time intervals during which the test time point is varied.
[0021] This circuitry may be associated with any entity or multiple entities configured to perform time-of-flight measurements, such as direct ToF, based on which a round-trip delay can be determined.
[0022] The circuitry may be based on or include one or more processors (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)), an FPGA (Field Programmable Gate Array), a ToF camera, a computer, a server, etc.
[0023] The circuitry may be adapted to control the light source and / or the imager, although in other embodiments the circuitry may only evaluate the results, i.e. in such embodiments the light source and / or the imager may be controlled by another entity.
[0024] In general, the light source may be configured to emit pulsed light and may be based on laser technology, such as VCSEL (Vertical Cavity Surface Emitting Laser), laser diode, or LED (Light Emitting Diode) technology. The light source may include a number of light emitting elements that may be arranged in an array. Some light emitting elements of the light source may be in a relationship with some imaging elements of the ToF image sensor, and the relationship may be any relationship, but the present disclosure is not limited to such a case.
[0025] The imaging element may be based on SPAD technology (single photon avalanche diode) or any other technology that may allow single photon counting, such as avalanche photodiodes (APDs) (discussed below) that may be correspondingly biased.
[0026] According to the present disclosure, the round trip delay can be measured directly, i.e., the time can be determined directly by the ToF circuitry indicating the distance a photon travels from the light source and back to the ToF imager.
[0027] To perform such measurements, a test number of photons can be determined at a test time point. The test time point can be any time point (in a measurement time interval (e.g., frame)) at which it is tested whether an emitted photon is detected by the imager. The test time point can be determined based on rough distance assumptions, or based on a time sweep, etc. Thus, the test time point can vary from measurement time interval to measurement time interval.
[0028] For example, if no photons are detected at the test time point, the test time point may be a different one in the next frame (or the same frame to ensure that no light is detected at the next measurement time interval either).
[0029] Furthermore, in some embodiments, a reference photon count may be determined at a reference time point. The reference time point may be any time point (in the measurement time interval) at which it is assumed that no emitted photons are detected by the imager. Thus, the distance between the test time point and the reference time point may be appropriately high.
[0030] According to the present disclosure, the test time point may precede the reference time point in each measurement time interval, or vice versa, and the order of the test and reference time points may vary from measurement time interval to measurement time interval.
[0031] However, it is not necessary for the test time point to change at each measurement time interval; it may be sufficient to change it only once (ie, different for the two measurement time intervals).
[0032] In some embodiments, the test number of photons is compared to a reference number of photons, whereby the round trip delay of the emitted photons can be determined.
[0033] For example, if within one frame the test number of photons is higher (a predetermined number of times) than the reference number of photons, it can be determined that the test time (with respect to the emission time) corresponds (approximately) to the round trip delay (wherein a calibration may be performed for isolated effects such as electronics). However, multiple measurement time intervals may be taken into account that the test number of photons should be higher than the test reference number of photons.
[0034] For example, if single photons are counted, the test time point may be determined to correspond to the round trip delay when the test number of photons is 1 and the reference number of photons is zero. If such a relationship is determined over multiple (consecutive) measurement time intervals, the test time point (relative to the emission time point) may be indicative of the round trip delay.
[0035] In general, the test time points and time reference points may indicate time intervals (e.g., test time intervals and reference time intervals) and may be generated by controlling the gate over very short (e.g., picosecond range) time periods (time intervals). The length of the time intervals may be short such that it is possible to detect only one photon (or any other predetermined number of photons) in each time interval. Thus, it will be understood by those skilled in the art that the term "time point" refers to a (very short) "time interval", although the term "time point" is used herein.
[0036] In some embodiments, as discussed herein, the test number of photons is indicative of the photons emitted. Note that in some embodiments, the test number of photons may not represent all of the photons emitted, but may indicate that the emitted photons were detected at the time of the test.
[0037] In some embodiments, the reference number of photons represents ambient light.
[0038] In general, ambient light may not be the only source of light when photons are detected at a reference point in time. For example, statistical noise (e.g., sensor noise, dark counts, etc.) may be such a source. However, it should be noted that the present disclosure may be applicable when ambient light exceeds a certain threshold, for example, under such conditions (e.g., bright daylight) under which known dToF systems may typically not be able to operate properly.
[0039] In some embodiments, the test number of photons is zero or one.
[0040] This may be the case when single photons are counted, and may also be true for the reference number of photons, as described above.
[0041] In some embodiments, the test time point is based on a test gating signal that constitutes a test time interval, and the reference point for this time point is based on a reference gating signal that constitutes a reference time interval, as discussed herein.
[0042] In some embodiments, the time-of-flight circuitry is further configured to modify the confidence value based on a comparison of the test number of photons to the reference number of photons.
[0043] As mentioned above, in some embodiments, one measurement time interval may not be sufficient to ensure that the test time point actually represents a round trip delay. This may be the case when single photons are counted. However, such a scheme can also be used for multiple photons being counted.
[0044] In such an embodiment, a confidence value may be defined, which may be preset to a particular value, and if the confidence value exceeds a predetermined threshold, a test time point may be determined to represent the round trip delay.
[0045] If there are photons counted at the test time point, the confidence may be increased by a predetermined amount, and if photons are counted at the reference time point, the confidence may be decreased by a predetermined amount (or a different amount depending on whether the two events are considered equally important). Thus, if at both time points one or no photons are counted, the net value of the confidence after the measurement time interval remains the same as before (although within the measurement time interval it may be continuously increased and decreased, which may depend on the particular implementation).
[0046] Thus, in some embodiments, the time-of-flight circuit is further configured to increase the confidence value if the test number of photons is greater than the reference number of photons, and / or decrease the confidence value if the reference number of photons is greater than the test number of photons.
[0047] In some embodiments, the time-of-flight circuitry is further configured to determine a round-trip delay if the confidence value exceeds a predetermined threshold, as discussed herein.
[0048] In some embodiments, the time-of-flight circuitry is further configured to change the test time point if the test number of photons falls below a predetermined threshold.
[0049] For example, the test time points may be assumed to represent round trip delays for a given number of measurement time intervals. If it is determined that this is not the case (i.e., based on a confidence level), then the test time points may be varied. However, the present disclosure is not limited to that case, as the variation in the test time points may be based on, for example, a sweep, as laid out above. Also, a mixture of different methods of varying the test time points may be used.
[0050] In some embodiments, the time-of-flight imager is based on a single-photon avalanche diode, as discussed herein.
[0051] In some embodiments, the ToF circuitry further includes a delay generator for generating the test time point and / or the reference time point.
[0052] There are many ways in which the delay generator may operate according to this disclosure. It may, for example, be swept linearly (or non-linearly) through the complete distance (or time) range, moving closer, far to far away, or far to closer. Depending on the distance, the step width may be modulated. For example, at longer distances, the steps may be smaller between successive distances. When an object is farther away, the time-of-flight photon flux is smaller, and by making the steps smaller, weaker signals are less likely to step over. It is also possible to fix the distance at, for example, 100, to obtain an accurate time-of-flight distance to within about one percent.
[0053] In some embodiments, when all distances have been tried, the search process continues, either by doing the same sequence or, for example, by doing the opposite sequence. In some cases, a random or pseudorandom sequence is applied to the search space. It is also possible that, for optimization purposes, a signal is given to the delay generator via an additional input port (not shown) for modification by the DSP. For example, it may be assumed that only a limited portion of the distances is searched. It is further possible that very short distances are skipped by default to avoid locking reflections in lenses shared between the outgoing and incoming beams.
[0054] In some embodiments, the boundary between matching and no matching can be adapted. For example, during a period of time, an excessive number of ambient photons may be incident during a measurement period (compared to a reference period), delaying the advance to the next trial distance. When the confidence value is read at that moment, it may be just above a predetermined voltage value (e.g., 500 millivolts), but not enough above the predetermined voltage to determine the round trip delay. Thus, depending on the application and / or the level of ambient photons, the boundary can be adapted. When the ToF light level is relatively low (or the distance is relatively long) (relatively: depending on the normal application), the system may not perform well. However, according to the present disclosure, in uncertain situations, the confidence level can be kept low enough.
[0055] In some embodiments, an offset is applied to distinguish between a match and a mismatch. The offset may correspond to a type of imbalance that must be overcome before the presence of a match is determined. According to the present disclosure, there are several ways to generate such an offset. The first is to make the reference gate (e.g., a reference time interval or reference time point discussed herein, see e.g., description of Figs. 1-3) wider, e.g., 10 percent, than the test gate (e.g., a test time interval or test time point discussed herein, see e.g., description of Figs. 1-3). The result is more counts on average downwards when the expected distance does not match the actual distance. It may generate a threshold that requires an excess amount of ambient photons to be outnumbered by the available time-of-flight photons in the test gate window in case of a match. The absolute number of time-of-flight events required can then be scaled proportionally to the ambient photon level. Another option for generating the offset may include setting the threshold at a certain level. Yet another option may include the timing generator generating a different number of test gates and ref gates to generate the imbalance.
[0056] A further option is to use a different value to increase reliability than to decrease it.
[0057] For example, when using a switched-capacitor circuit (as described below), if the reference period is larger than the test period (e.g., 10 percent larger, so that in the case of 500 millivolts it could be changed to 4 to 50 millivolts), an imbalance could be introduced to make the ToF system more stable. Such an embodiment may not require additional circuitry within the pixel, and such an embodiment may be applicable to an entire pixel array.
[0058] All or some of these options may be combined as well.
[0059] In some embodiments, the averaging level is modified, for example it is averaged over a large number of events, for example over 1000 events, so that the ambient and time-of-flight events have a confidence level that does not go up or down too much under high ambient photon levels. For this, a second averaging filter, for example an FIR filter, an analog filter, etc., or a low-pass filter can be applied between the averaging means and the comparator.
[0060] In one embodiment, the time-of-flight circuit further includes a switched-capacitor averaging circuit including an averaging capacitor, a sampling capacitor, and a switch for switching between the averaging capacitor and the sampling capacitor to compare the test number of photons to a reference number of photons, as discussed with reference to FIG. 3.
[0061] Thus, in some embodiments, the time-of-flight circuit is further configured to control a switch to switch between the averaging capacitor and the sampling capacitor.
[0062] In some embodiments, the switched capacitor averaging circuit is further configured to perform exponential moving averaging based on a ratio between an averaging capacitor capacitance (the capacitance of the averaging capacitor) and a sampling capacitor capacitance (the capacitance of the sampling capacitor).
[0063] Some embodiments relate to a time-of-flight method for determining the round trip delay of photons emitted by a light source and incident on a time-of-flight imaging element, the time-of-flight method comprising the steps of measuring a test number of photons at a test time point and a reference number of photons at a reference time point, as discussed herein, and comparing the test number of photons with the reference number of photons to determine the round trip delay of the emitted photons, the test time point and the reference time point being included in one measurement time interval of at least two measurement time intervals during which the test time point is varied.
[0064] This TOF method can be implemented by a ToF circuit according to the present disclosure.
[0065] In some embodiments, the test number of photons represents emitted photons, as discussed herein. In some embodiments, the reference number of photons represents ambient light, as discussed herein. In some implementations, the test number of photons is zero or one, as discussed herein. In some embodiments, the test time point is based on a test gating signal that constitutes a test time interval, and the reference point of this time point is based on a reference gating signal that constitutes a reference time interval, as discussed herein. In some embodiments, the time of flight further comprises modifying the confidence value based on a comparison of the test number of photons to the reference number of photons, as discussed herein. In some embodiments, the time of flight method further comprises increasing the confidence value if the test number of photons is greater than the reference number of photons, and / or decreasing the confidence value if the reference number of photons is greater than the test number of photons, as discussed herein. In some embodiments, the time-of-flight method further comprises determining a round trip delay if the confidence value exceeds a predetermined threshold, as discussed herein. In some embodiments, the time-of-flight method further comprises varying the test time point if the test number of photons is below a predetermined threshold, as discussed herein. In some embodiments, the time-of-flight imager is based on a single-photon avalanche diode, as discussed herein. In some embodiments, the time-of-flight method comprises switching between an averaging capacitor and a sampling capacitor to compare the test number of photons to a reference number of photons, as discussed herein. In some embodiments, the time-of-flight method comprises controlling a switch for switching between the averaging capacitor and the sampling capacitor, as discussed herein. In some embodiments, the time-of-flight method comprises performing an exponential moving averaging based on a ratio between the averaging capacitor capacitance and the sampling capacitor capacitance, as discussed herein.
[0066] The methods described herein may further be implemented as a computer program that, when executed by a computer and / or processor, causes the computer and / or processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided that stores a computer program product that, when executed by the above-mentioned processor, causes the method described herein to be performed.
[0067] Some embodiments relate to a ToF receiver for estimating distance including a SPAD circuit for generating an event upon detection of a photon, a switch control circuit for generating a first signal for sampling a modulation voltage and a second signal for including the sample in an average output, and an averaging demodulator having a demodulation voltage applied to its input node responsive to the first signal configured to sample the demodulation voltage in response to a second signal including the sample in an output voltage on one node for estimating TOF distance.
[0068] Returning to Figure 1, there is depicted a timing diagram 1 according to the present disclosure. Additionally, Figures 2 and 3 depict exemplary embodiments of ToF systems 10 (Figure 2) and 20 (Figure 3) including ToF circuitry 130 (Figure 2) and 140 (Figure 3) according to the present disclosure, based on which the timing diagrams will be further described.
[0069] Timing FIG. 1 shows signal versus time to illustrate the principles of the present disclosure. Successive measurement cycles are performed, each beginning with a light pulse 200 emitted by a laser 100 (or an LED in other embodiments) into a scene (not shown). Between these periods 280 there are temporal pauses 281, demonstrating that in this embodiment the measurement periods are not adjacent in time. However, in other embodiments at least a subset of the measurement cycles are adjacent. The breaks 281 have the same length in this embodiment, but may generally be of a constant or variable nature.
[0070] Reasons for making them variable include lowering power dissipation, and randomizing the emitted output light to allow multiple 3D camera systems to operate on the same scene, rendering the laser output of another camera to be interpreted as uncorrelated with its own laser output, similar to ambient light, and thereby reducing interference. The breaks may be of any length necessary to lower power consumption.
[0071] Photons 205 are incident on the SPAD receiver 120. Some photons originate from ambient light on the scene and some are present due to the pulsed illumination 200 being reflected onto the scene.
[0072] For example, ambient light photons may be 10-1000 times more powerful than emitted photons (even after spectral filtering) without the disclosure being limited in this respect.
[0073] The gating signal 111 is configured in time shown as curve 240. The gating signal 111 includes two types of gating windows that can appear in the measurement cycle 280. A gating window is a short period during which the SPAD receiver 120 is called upon to check if there is a photon incident. If so, the SPAD receiver 120 signals via its output 121 that a photon has incident, otherwise it keeps its output 121 silent. Also, a dark count signal can occur during the window and has similar characteristics as the ambient light photon and dark counts and dark count rate (DCR), and therefore will be ignored in the subsequent text.
[0074] A test delay period 210 is determined, which represents the expected distance to the scene. This test delay period 210 points to the start of a measurement cycle 280, which starts with the light pulse 200. After the test delay period 210, a test gate window 220 (test time interval) opens for a short period (test time point) (in this embodiment, 50 picoseconds, but in general the test time interval can be any length, e.g., between 50 picoseconds and a few nanoseconds). The SPAD receiver 120 is asked to check whether a photon is incident during this test gate window 220.
[0075] A reference gate window 230 (reference time interval) is opened for a short period of time (reference time point) (in this embodiment, 50 picoseconds, but in general the test time interval may be any length, e.g., between 50 picoseconds and a few nanoseconds). The SPAD receiver 120 is then asked to check whether a photon falls into this reference gate window 220.
[0076] The gating event 121, curve 250, arising from the SPAD receiver 120 signals the presence of a photon during the window period during which the SPAD receiver 120 is required to be sensitive, ie, during the period during which the gating signal 111 is high.
[0077] The timing generator 110 is configured to generate the gating signal 111 and provide a signal up / down 112 that signals whether the gating window was the test gating window 220 or the reference gating window 230 .
[0078] When the assumed distance corresponding to the test delay period 210 does not match the actual distance, there are no time-of-flight photons within the test gate window 220, leaving only ambient photons to generate part of the gate event 121. By comparing the rate or number of events resulting from the test gate 220 with the reference gate 230, a comparable number / rate of events can be obtained, demonstrating the absence of light incident correlated with the outgoing light pulse 200.
[0079] If the assumed distance corresponding to the test delay period 210, however, matches the actual or approximate distance, then most of the incident time-of-flight photons 172 will be within the test gate window 220 and generate a portion of the gate events 121. By comparing the rate or number of events resulting from the test gate 220 to the reference gate 230, a significantly higher number / rate of events is obtained, demonstrating that light is incident and correlates with the emitted light pulse 200, demonstrating that the correct distance is being assumed and tested.
[0080] In FIG. 2, the comparison is performed by using an up / down counter 132, which counts up or down at each gating event 121 in response to the digital up / down signal 112 (curve 260 in FIG. 1).
[0081] Once a distance match is performed, the ambient photons in both test windows cancel each other out and the time-of-flight photon 172 generates a gating event only during the test gate 220, and as such, the counter 132 increments. This digital value, or a portion thereof, can be signaled to the DSP as the reliability signal 153. The counter can be a limited number of bits, including 6, 8, 10, 12, or 16, without limiting the disclosure in that respect.
[0082] In some embodiments, the counter is designed so that it cannot overflow, i.e., once it reaches its maximum positive value, it then remains limited to its maximum level (it does not collapse to the most negative value) once further upward counting is required. The threshold level 134 can be used to determine where the boundary is located for considering possible distance matches.
[0083] In FIG. 3, the comparison is performed by a switched capacitor averaging circuit that includes a switch 142, an averaging capacitor C_a, and a sampling capacitor C_s that generates an analog trust signal 156.
[0084] The threshold level 144 is used to determine where the boundary is located to consider possible distance matching. In the absence of matching, when the ascending / descending signal 112 is driven low (e.g., zero volts) by the timing generator 110, the same statistically number and rate of events will occur during the test gate 220 when the ascending / descending signal is high (e.g., 1 volt) as during the reference gate 230. Assuming C_a is n times larger than C_s, an exponential moving average is performed for the 1 volt and zero volt input signals. An average is then taken over the last C_a / C_s number of samples. If there is an average number of occurrences (i.e., no distance matching), then the average output voltage will be close to halfway between the high and low voltage values of the up / down signal 112 that make up the confidence signal 156. When the digital voltage values of the up / down signal 112 are 1V and 0V high and low, respectively, the average voltage will then be close to 500 millivolts.
[0085] Curve 270 (in FIG. 1) illustrates the update of the confidence level by a small amount, sometimes upwards (if the event is in the test gating window 220), and sometimes downwards (if the event is in the reference gating window 230).
[0086] In Figures 2 and 3, comparators 132 and 141, respectively, receive signal Next_OK 152 which remains low when there is an indication of a possible distance match and therefore are not authorized to step to the next hypothesized distance and therefore not to update test delay signal 161 and not to change test delay period 220.
[0087] However, if the Next_OK signal 152 goes high, the confidence of the match is deemed low enough to proceed to the next position. The clock 150 generates a signal 155 that defines the time moment at which the Next_OK signal 152 is evaluated, which can in effect stimulate the next assumed distance. In this case, the signal Next 154 passes this message to the pulse or edge-delay value generator 160 to update the test delay signal 161 used by the timing generator 110 to be applied for the updated dead delay 210 during the measurement. The number of measurement cycles 280 that pass before this evaluation can be made can be anywhere between hundreds to millions of measurement cycles. The higher the number of measurement cycles used to evaluate the confidence, the more the decision is considered, but the longer it takes before an update can be made. The delay value generator 160 is configured to generate an adapted hypothetical distance, which is reflected in an associated test delay signal 161, each time it receives a next signal 154. The timing generator 110 will use the latter signal to apply an associated test delay period 210. The delay value generator 160 proposes each time a next signal 154, a new test delay signal 161, is input at its input.
[0088] FIG. 4 gives a statistical simulation over a period of 1 million measurement cycles of 40 nanoseconds each for a total of 40 milliseconds. It is based on the analog principle 140 (FIG. 3) for generating reliability, with an average length C_a / C_s of 1600, an average ambient photon event rate of 1 photon per 200 nanoseconds, and an average of only 4 time-of-flight photons per 1000 measurement cycles (4 photons per 40 microseconds). The widths of the test gate and reference gate signals are both 500 picoseconds. The emitted light pulse FWHM (full width at half maximum) is 100 picoseconds. The dashed line 310 is the ground truth period, which starts at 10 nanoseconds and whose steps are from half to 22 nanoseconds. The simulation starts with a test delay period (curve 300) of 0 nanoseconds.
[0089] The Y-axis 390 goes from zero to 40 nanoseconds. The Y-axis 395 goes from 0 to 1 for the confidence signal 320. The time axis 380 goes from zero to 1 million measurement cycles, i.e. 40 milliseconds. If the test delay period 300 does not match the ground truth 310, the confidence moves around 0.5. In this case, incremental steps are made based on the decision clock 150, which has a frequency of 100 kilohertz, so that every 10 microseconds, a small step can be allowed in response to the Next_OK 152 signal. The used threshold 144 is set to 500 millivolts until halfway through. After a while, the test delay period 300 reaches the ground truth 310 where the confidence jumps, preventing further steps of the test delay 300. Then, at 1 million cycles, the ground truth is suddenly updated to 23 nanoseconds (emulating an object moving from place to place). From this point onwards, there are many counts above and below on average, bringing the confidence 320 to its intermediate value, sometimes it is above 500 millivolts and sometimes it is below. In periods where it is above 500 millivolts, the test delay period 300 stops increasing, if it is below it the increase continues. Once it reaches 20 nanoseconds the increase stops as the confidence level has reached 500 millivolts and so what was sought for the ground truth 310 has been found.
[0090] At transition 340, lock is obtained, and at transition 350, there is a ground truth change followed by a sudden drop in confidence; once the confidence drops below 500 millivolts, the test delay period 300 begins to increase again and once it reaches its final destination, transition 370, the confidence increases significantly because at that point there are again a large number of time-of-flight photons detected at the test gate 220 with the correct test delay period of 20 nanoseconds.
[0091] The blocks described in Figures 2 and 3 can be implemented in multiple ways according to the knowledge of those skilled in the art. Furthermore, the SPAD receiver may have different implementations. For example, an excess bias can be applied at the gate moment to keep the SPAD below its breakdown voltage during the remaining period. Incident photons outside the gate period will generate only minority carriers without causing breakdown, which can reduce the power dissipation by a large factor.
[0092] FIG. 5 illustrates a TOF method according to the present disclosure.
[0093] At 610, a first test delay is applied, after which a predetermined number of measurement cycles are performed at 620 and an evaluation period elapses (e.g., as determined by clock 150). During each measurement period, light is emitted, and after the test delay, a gate-window is opened and if a photon is detected, the confidence signal is increased. During a reference gate period, photons cause the confidence to decrease so that the rates and / or number of events during the gate period are compared.
[0094] At 630, it is determined whether the evaluation period has elapsed. If not, another measurement cycle is performed.
[0095] If yes, then at 640 it is checked whether the confidence is above a predefined threshold.
[0096] When the number of events into which time-of-flight photons can enter is significantly higher, a threshold is exceeded and the test delay is no longer updated.
[0097] FIG. 6 shows a flip-flop 400 for estimating object distance.
[0098] In general, the digital signal processor can use the value of the test delay, or any derived value, as a measure of object distance. Once it has read the latest value, it sets the flip-flop 400 through a reset signal 420, so that the output bit 410 indicating whether a "change has occurred" is set low (e.g., "0"). If the object distance changes, the next 154 goes high, thereby setting the output bit indicating whether a change has occurred 410, thus, for example, high ("1"). In this way, the DSP can monitor whether one or more distances (from multiple systems 198, 199) have changed at low power.
[0099] The circuits 10 and / or 20 can be repeated in 1D or 2D arrays to construct an image sensor without limiting the disclosure in that respect. Those skilled in the art of arrays can choose to keep certain parts common (e.g., clock generation 150). Also, flip-flop 400 can be instantiated for each pixel or for groups of pixels to signal to a DSP processor that a substantial change has possibly triggered a higher speed measurement cycle 280 and possibly other operations at the processing level. Readout transistors as known in the art can be added to arrange the array operation to suit the particular envisioned application.
[0100] FIG. 7 illustrates in block diagram form a TOF embodiment according to the present disclosure.
[0101] At 701, a test number of photons is determined at a test time point and a reference number of photons is determined at a reference time point, as discussed herein.
[0102] At 702, the test number is compared to a reference number, as discussed herein.
[0103] FIG. 8 illustrates in block diagram form a TOF embodiment according to the present disclosure, where confidence values are further determined.
[0104] At 801, a test number of photons is determined at a test time point, as discussed herein, and a reference number of photons is determined at a reference time point. At 802, the test number is compared to the reference number, as discussed herein.
[0105] At 803, the confidence value is modified depending on the comparison result, as discussed herein. In this embodiment, the confidence value is modified based on an up / down counter. If the test number of photons is greater than the reference number of photons, the confidence value is increased, and if the reference number of photons is less than the test number of photons, the confidence value is decreased.
[0106] At 804, if the confidence value exceeds a predetermined threshold, the round trip delay is determined, otherwise the test time point is varied based on a sweep as discussed herein.
[0107] Any of the systems presented herein or based thereon can be complemented by other means known in the state of the art of image sensors. For example, microlenses, color filters, etc. may be applied to optimize the light input to the single photon detection circuit. Any means to improve the internal / external quantum efficiency, responsivity, and detection probability may be applied. Three-dimensional stacking may be performed, for example, the SPAD detector layer may start from another wafer / material, then the CMOS circuit wafer. Backside illumination (BSI) may be applied, current assist may be applied, or silicon-on-insulator (SOI) technology may be applied. The proposed embodiments of the present disclosure may be laid out as pixels for a sensor array in making a 3D image sensor in total. Some signals may be grouped for multiple pixels or may be the same for the whole array, such as those that define the window, the demodulation function, and the signal that determines the averaging length n. In addition, standard 3T or 4T image sensor pixels can be added for simultaneous standard image sensing. The SPAD receiver 120 can include a regular SPAD, but can also include any other means for achieving single photon detection, including an avalanche photodetector (APD) with sufficient gain, utilizing its linear gain mode to operate the diode below breakdown and still achieve digital photon arrival edges and events.
[0108] It should be understood that the above-described embodiments describe methods with exemplary ordering of method steps. However, the particular ordering of method steps is provided for illustrative purposes only and should not be construed as binding.
[0109] It should be noted that the division of the ToF system 10 or 20 into units 101 to 156 is for illustrative purposes only, and the present disclosure is not limited to any particular division of functionality in any particular unit. For example, the ToF systems may be implemented by respective programmed processors, field programmable gate arrays (FPGAs), etc.
[0110] The methods described herein may also be implemented as a computer program that, when executed by a computer and / or processor, causes a computer and / or processor to perform the method. Also, in some embodiments, a non-transitory computer-readable recording medium is provided that stores a computer program product that, when executed by a processor, such as the processors described above, causes the above-described methods to be performed.
[0111] All units and entities described in this specification and claimed in the accompanying claims may be implemented as integrated circuit logic, for example on a chip, unless otherwise stated, and the functions provided by such units and entities may be implemented by software, unless otherwise stated.
[0112] In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be understood that computer programs providing such software control, and the transmission, storage, or other media on which such computer programs are provided, are contemplated as aspects of the present disclosure.
[0113] The present technology can also be configured as follows. (1) A time-of-flight circuit for determining a round trip delay of a photon emitted by a light source and incident on a time-of-flight imager, the time-of-flight circuit comprising: measuring a test number of photons at a test time point and a reference number of photons at a reference time point; comparing the test number of photons to the reference number of photons to determine the round trip delay of the emitted photons. It is structured as follows: The test time point and the reference time point are included in one measurement time interval of at least two measurement time intervals during which the test time point is varied. Time-of-flight circuit. (2) the test number of photons indicates the photons emitted; A time-of-flight circuit as described in (1). (3) The reference number of the photon represents the ambient light. A time-of-flight circuit according to (1) or (2). (4) The test number of said photons is zero or one. A time-of-flight circuit according to any one of (1) to (3). (5) The test time point is based on a test gating signal constituting a test time interval, and the reference time point is based on a reference gating signal constituting a reference time interval. The time-of-flight circuit according to any one of (1) to (4). (6) modifying a confidence value based on a comparison of the reference number of photons to the test number of photons; It is further configured as follows: A time-of-flight circuit according to any one of (1) to (5). (7) increasing the confidence value if the test number of photons is greater than the reference number of photons; or If the reference number of photons is greater than the test number of photons, then decrease the confidence value. It is further configured as follows: (6) A time-of-flight circuit as described in (6). (8) determining a round trip delay if the confidence value exceeds a predetermined threshold; It is further configured as follows: A time-of-flight circuit according to (6) or (7). (9) changing the test time if the test number of photons is below a predetermined threshold; It is further configured as follows: A time-of-flight circuit according to any one of (1) to (8). (10) The time-of-flight imager is based on a single-photon avalanche diode. The time-of-flight circuit according to any one of (1) to (9). (11) further comprising a switched-capacitor averaging circuit including an averaging capacitor, a sampling capacitor, and a switch for switching between the averaging capacitor and the sampling capacitor for comparing the test number of photons with the reference number of photons. The time-of-flight circuit according to any one of (1) to (10). (12) The switched capacitor averaging circuit is configured to perform exponential moving averaging based on a ratio between an averaging capacitor capacitance and a sampling capacitor capacitance. A time-of-flight circuit as described in (11). (13) A time-of-flight method for determining a round trip delay of a photon emitted by a light source and incident on a time-of-flight imager, the method comprising: Measuring a test number of photons at a test time point and a reference number of photons at a reference time point; comparing the test number of photons to the reference number of photons to determine a round trip delay of the emitted photons. Including, The test time point and the reference time point are included in one measurement time interval of at least two measurement time intervals during which the test time point is varied. Flight time method. (14) The test number of photons indicates the photons emitted. (13) A time-of-flight method as described in (13). (15) The reference number of the photon indicates the ambient light. A time-of-flight method according to (13) or (14). (16) The test number of photons is zero or one. The time-of-flight method according to any one of (13) to (15). (17) The test time point is based on a test gating signal constituting a test time interval, and the reference time point is based on a reference gating signal constituting a reference time interval. The time-of-flight method according to any one of (13) to (16). (18) modifying a confidence value based on a comparison of the reference number of photons to the test number of photons. It further includes The time-of-flight method according to any one of (13) to (17). (19) increasing the confidence value if the test number of photons is greater than the reference number of photons; or If the reference number of photons is greater than the test number of photons, then decrease the confidence value. It further includes (18) A time-of-flight method. (20) determining a round trip delay if the confidence value exceeds a predetermined threshold; It further includes A time-of-flight method according to (18) or (19). (21) changing the test time if the test number of photons is below a predetermined threshold; It further includes The time-of-flight method according to any one of (13) to (20). (22) The time-of-flight imager is based on a single-photon avalanche diode. The time-of-flight method according to any one of (13) to (21). (23) switching between an averaging capacitor and a sampling capacitor to compare the test number of photons to the reference number of photons. The time-of-flight method according to any one of (13) to (22). (24) The method further includes performing exponential moving averaging based on a ratio between the averaging capacitor capacitance and the sampling capacitor capacitance. (23) A time-of-flight method according to (23). (25) A computer program comprising a program code that, when executed on a computer, causes the computer to execute the method according to any one of (11) to (20). (26) A non-transitory computer-readable recording medium having stored therein a computer program product that, when executed by a processor, causes the computer to perform any one of the methods described in (11) to (20).
Claims
1. 1. A time-of-flight circuit for determining a round trip delay of photons emitted by a light source and incident on a time-of-flight imager, the time-of-flight circuit comprising: measuring a test number of photons at a test time point and a reference number of photons at a reference time point; comparing the test number of photons to the reference number of photons to determine the round trip delay of the emitted photons. It is structured as follows: The test time point and the reference time point are included in one measurement time interval of at least two measurement time intervals during which the test time point is varied. Time-of-flight circuit.
2. and a switched capacitor averaging circuit including an averaging capacitor, a sampling capacitor, and a switch for switching between the averaging capacitor and the sampling capacitor for comparing the test number of photons with the reference number of photons.
2. The time-of-flight circuit of claim 1.
3. The switched capacitor averaging circuit is configured to perform exponential moving averaging based on a ratio between an averaging capacitor capacitance and a sampling capacitor capacitance.
3. The time-of-flight circuit of claim 2.
4. The test number of photons is zero or one 2. The time-of-flight circuit of claim 1.
5. The test time points are based on a test gating signal that constitutes a test time interval, and the reference time points are based on a reference gating signal that constitutes a reference time interval.
2. The time-of-flight circuit of claim 1.
6. Varying a confidence value based on a comparison of the reference number of photons and the test number of photons. It is further configured as follows:
2. The time-of-flight circuit of claim 1.
7. increasing the confidence value if the test number of photons is greater than the reference number of photons; or If the reference number of photons is greater than the test number of photons, then decrease the confidence value. It is further configured as follows:
7. The time-of-flight circuit of claim 6.
8. If the confidence value exceeds a predetermined threshold, then determining the round trip delay It is further configured as follows:
7. The time-of-flight circuit of claim 6.
9. If the test number of photons is below a predetermined threshold, then the test time is changed. It is further configured as follows:
2. The time-of-flight circuit of claim 1.
10. The time-of-flight imager is based on a single-photon avalanche diode 2. The time-of-flight circuit of claim 1.
11. 1. A time-of-flight method for determining the round trip delay of photons emitted by a light source and incident on a time-of-flight imager, the method comprising: Measuring a test number of photons at a test time point and a reference number of photons at a reference time point; comparing the test number of photons to the reference number of photons to determine a round trip delay of the emitted photons. Including, The test time point and the reference time point are included in one measurement time interval of at least two measurement time intervals during which the test time point is varied. Flight time method.
12. and switching between an averaging capacitor and a sampling capacitor to compare the test number of photons to the reference number of photons.
12. The time-of-flight method of claim 11.
13. performing exponential moving averaging based on a ratio between the averaging capacitor capacitance and the sampling capacitor capacitance.
13. A time-of-flight method according to claim 12.
14. The test number of photons is zero or one 12. The time-of-flight method of claim 11.
15. The test time points are based on a test gating signal that constitutes a test time interval, and the reference time points are based on a reference gating signal that constitutes a reference time interval.
12. The time-of-flight method of claim 11.
16. Varying a confidence value based on a comparison of the reference number of photons and the test number of photons. It further includes 12. The time-of-flight method of claim 11.
17. increasing the confidence value if the test number of photons is greater than the reference number of photons; or If the reference number of photons is greater than the test number of photons, then decrease the confidence value. It further includes 17. A time-of-flight method according to claim 16.
18. If the confidence value exceeds a predetermined threshold, then determining the round trip delay It further includes 17. A time-of-flight method according to claim 16.
19. If the test number of photons is below a predetermined threshold, then the test time is changed. It further includes 12. The time-of-flight method of claim 11.
20. The time-of-flight imager is based on a single-photon avalanche diode 12. The time-of-flight method of claim 11.
Citation Information
Patent Citations
Photoelectric conversion element, light receiving device, light receiving system, and distance measuring device
JP2011080874A
Method and system for high resolution long range flash LIDAR
JP2021513087A
Method for measuring a time of flight
US20160047904A1
Avalanche diode based object detection device
US20200088843A1
Imaging systems with single-photon avalanche diodes and ambient light level detection
US20210223398A1
