Time-of-flight circuit and time-of-flight readout method - Patents.com

The time-of-flight circuit with time-shifted readout taps addresses SNR and tap mismatch issues in ToF sensors, improving efficiency and reducing complexity by employing multiplexing techniques.

JP2024543594AActive Publication Date: 2024-11-21SONY SEMICON SOLUTIONS CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2024533001
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-09
Filing Date
2022-11-14
Publication Date
2024-11-21
Estimated Expiration
2042-11-14

AI Technical Summary

Technical Problem

Existing time-of-flight (ToF) devices face challenges in achieving a favorable signal-to-noise ratio (SNR) and dealing with tap mismatch, particularly in indirect ToF (iToF) sensors, while also desiring to reduce design complexity and increase dynamic range.

Method used

Implementing a time-of-flight circuit with a plurality of time-shifted readout taps applied to the outputs of an imaging device, where the taps are temporally shifted to read out the outputs one after another, utilizing techniques such as charge cycling and delta counting to mitigate tap mismatch and improve SNR.

Benefits of technology

This approach enhances the SNR and dynamic range of ToF sensors, reduces design complexity, and optimizes silicon area usage by employing spatial or temporal multiplexing methods, making mature iToF and dToF technologies more efficient.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543594000001_ABST
    Figure 2024543594000001_ABST
Patent Text Reader

Abstract

The present disclosure relates generally to a time-of-flight circuit comprising an imager and a readout circuit configured to apply a plurality of time-shifted readout taps to a plurality of outputs of the imager, the time-shifted readout taps being shifted in time with respect to one another such that the plurality of outputs of the imager are read out one after the other.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure generally relates to time-of-flight circuits and time-of-flight readout methods. [Background technology]

[0002] Generally, Time-of-Flight (ToF) devices are known for determining distances to a scene (eg, an area, an object, etc.).

[0003] For example, a distinction can be made between iToF (indirect ToF) and dToF (direct ToF), which differ in their approach to determining distance.

[0004] In dToF, distance can be measured directly by measuring the round-trip delay of a light pulse from emission to detection after being reflected by the scene.

[0005] In dToF, multiple counters can be used that can count the multiplied electronic signals generated based on the light incident (reflected) on the image sensor. The counters can be time-synchronized so that a histogram can be generated and the distance can be concluded based on the peak of the histogram.

[0006] In iToF, distance may be measured based on a modulated signal. For example, the modulated signal may be used to modulate a light source such that modulated light is emitted. The modulated signal may be reused and regenerated such that multiple modulated signals are applied to the output of the image sensor, where the modulated signal may be further phase shifted. Based on the overlap of the detected returning modulated light signal and the phase-shifted modulated signal, distance can be determined.

[0007] While techniques exist for performing distance measurements, it is generally desirable to provide time-of-flight circuitry and a time-of-flight readout method. Summary of the Invention

[0008] According to a first aspect, the present disclosure provides a time-of-flight circuit comprising an imager and a readout circuit configured to apply a plurality of time-shifted readout taps to a plurality of outputs of the imager, the time-shifted readout taps being shifted in time with respect to each other such that the plurality of outputs of the imager are read out one after the other.

[0009] According to a second aspect, the present disclosure provides a time-of-flight readout method comprising applying a plurality of time-shifted readout taps to a plurality of outputs of an imager, the time-shifted readout taps being shifted in time with respect to each other such that the plurality of outputs of the imager are read out one after the other.

[0010] Further aspects are set out in the dependent claims, the following description and the drawings. [Brief description of the drawings]

[0011] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1A] 1 illustrates one embodiment of a ToF circuit implemented as a spatially multiplexed Ntap sensor. [Figure 1B] 1B illustrates one embodiment of a timing diagram for signals in the ToF circuit of FIG. 1A. [Figure 2A] 1 illustrates an embodiment of a ToF circuit that includes an imaging element having nine imaging sub-elements based on a SPAD. [Figure 2B] 2B illustrates an embodiment of a timing diagram for the ToF circuit of FIG. 2A. [Figure 3A] 1 shows two embodiments of a ToF circuit to illustrate the principles of a temporally multiplexed Ntap sensor. [Figure 3B] 3B illustrates an embodiment of a timing diagram for the right ToF circuitry of FIG. 3A. [Figure 4]4A illustrates an embodiment of a ToF circuit having multiple counters; FIG. 4B illustrates an embodiment of a timing diagram for the ToF circuit of FIG. 4A; and FIG. 4C illustrates an embodiment of a histogram generated based on the ToF circuit of FIG. 4A. [Figure 5A-C] 5A shows an embodiment of a ToF circuit including multiple bidirectional counters; FIG. 5B shows an embodiment of a timing diagram for the ToF circuit of FIG. 5A; and FIG. 5C shows an embodiment of a timing diagram for the counters of FIG. 5A. [Figure 5D] 5B illustrates one embodiment of a delta histogram generated based on the ToF circuitry of FIG. 5A. [Figure 6] 1 illustrates a ToF readout method according to the present disclosure, where multiple readout taps are applied to multiple image sub-elements. [Figure 7] A further embodiment of the ToF readout method according to the present disclosure, where multiple time-shifted readout taps are applied to multiple capacitances of the imager. [Figure 8] 1 illustrates a further embodiment of a ToF readout method according to the present disclosure, where multiple time-shifted readout taps are applied to multiple counters. [Figure 9] 13 illustrates a further embodiment of a ToF readout method according to the present disclosure, in which a histogram is generated based on multiple counters. [Figure 10] 1 illustrates an embodiment of a ToF imaging system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Before a detailed description of the embodiment is given, starting with FIG. 1, a general description will be given.

[0013] ToF devices as described above are generally known, however, it has been recognized that N-tap operation may be utilized to improve the trade-off between signal-to-noise ratio (SNR) and anti-aliasing range (in iToF).

[0014] N-tap operation may refer to the use of multiple taps (e.g., eight or more (or less)) instead of two taps, as is known for iToF, so that an iToF sensor can rival a dToF sensor in terms of SNR under high ambient light.

[0015] However, the N-tap is not limited to the iToF case, as the principles of the present disclosure may be applied to a dToF sensor or dToF readout circuit that is detected based on a modulated signal or tap, for example by detecting pulsed light.

[0016] Additionally, it has been recognized that it may be desirable to reduce / mitigate tap mismatch, for example, by using charge cycling techniques or by using multiple taps per pixel as described herein.

[0017] It has further been recognized that it is desirable to increase dynamic range (and / or area efficiency) and conserve silicon area by using fewer bit-width counters, e.g., based on delta counting techniques, as described herein.

[0018] In general, the principles of the present disclosure may be described as applied to stacked sensor configurations (e.g., where the photosensor layer and readout / modulation circuitry may be on separate wafers), but, if necessary, the principles of the present disclosure may be adapted accordingly to non-stacked configurations.

[0019] Additionally, it has been recognized that it would be desirable to provide an N-tap sensor with lower design complexity by implementing the N-tap in a spatially or temporally multiplexed manner in analog or digital based circuitry, thereby also allowing the use of mature iToF or dToF technology.

[0020] Accordingly, some embodiments relate to a time-of-flight circuit comprising an imager and a readout circuit configured to apply a plurality of time-shifted readout taps to a plurality of outputs of the imager, the time-shifted readout taps being shifted in time with respect to each other such that the plurality of outputs of the imager are read out one after the other.

[0021] Although the ToF circuit may include more elements than the image sensor and the readout circuit, these two elements may be sufficient to implement the principles of the present disclosure, such as corresponding connections and electronic elements, which can be envisaged according to the ordinary skill of a person skilled in the art.

[0022] The imager may be based on any imaging technology suitable for performing ToF measurements, such as diode technology, gate technology, demodulation technology, etc. Thus, the imager may be based on at least one of the following: SPAD (Syndiphoton Avalanche Diode), APD (Avalanche Photodiode), CAPD (Current Assisted Photonic Demodulator), Gated ToF Pixel, Photon Multiplier, or any other imager with (intrinsic) electron multiplication function or gain, etc.

[0023] The imager may include multiple outputs corresponding to the readout taps.

[0024] For example, if the imaging element is based on diode technology (note that this disclosure is not limited to SPADs and ADPs; any diode or electron multiplication technology may be envisioned based on which photons can be counted), the output may refer to a connection that includes or can connect to a photon counter or register.

[0025] In another example, if the imager is based on CAPD or gate technology, the output may correspond to a gate or tap (with a specific doping, as it is commonly known).

[0026] The readout circuitry may thus be any circuitry that can be used to read out such an output, i.e. a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), any type of integrated circuit (IC) that can be used to apply the readout tap, etc., and it may be envisaged that a plurality of such entities may also be combined using corresponding interfaces etc.

[0027] According to the present disclosure, multiple readout taps can be applied to the output of the imager. In some embodiments, the readout taps refer to functional entities and thus may not be limited to, for example, CAPD taps. In some embodiments, the readout taps refer to signals that are applied to the respective outputs for readout from the outputs.

[0028] As indicated above, the read taps may be time shifted relative to one another. For example, each read tap may be applied after another with a predetermined time (interval) between the application of each read tap. The predetermined time may be different for each read tap or may be the same (roughly) for each read tap.

[0029] In some embodiments, the time shift between read taps is equal for each read tap, however, in some embodiments the time shifts need not be equal.

[0030] In some embodiments, a read tap is applied after a predetermined time during which the previous read tap is no longer applied.

[0031] In some embodiments, a read tap is applied immediately after a previous read tap.

[0032] Thus, in some embodiments, the time-shifted read taps are time-shifted relative to each other, as described above, and it can be envisioned to apply different types of read taps "relative to each other," as described above.

[0033] Further, in an embodiment, the time-shifted readout taps shift the output of the imager so that they are read out one after the other.

[0034] For example, a first (electrical) charge (e.g., electrons, holes, electron-hole pairs, etc.) may be collected (from the imager or from within the imager) at a first output (either within the first output or from a predefined area about the first output). After the first charge is collected and read out, a second charge is collected and read out, etc.

[0035] In some embodiments, the charges may be collected simultaneously at the output, so that if the read taps are applied one after the other (e.g., without overlap in time), the outputs, and therefore the charges, can be read out one after the other.

[0036] In some embodiments, the charges may be generated in the same area (e.g., photosensitive area) of the imager, and corresponding readout taps may pull the charges toward their respective outputs. Thus, if the readout taps are time shifted so that they do not overlap, a first charge generated for a first time interval is read out at a first readout tap, a second charge generated for a second time interval is read out at a second readout tap, and so on.

[0037] In some embodiments, the number of read taps corresponds to the number of outputs, although these numbers may also vary.

[0038] For example, if all outputs are to be read by a readout circuit, the number of readout taps should be at least as large as the number of outputs. These numbers may be equal, but the number of readout taps may be greater than the number of outputs. For example, it may be envisaged that (one or several) particular outputs or all outputs are read by more than one readout tap (for example, in a demodulation scheme known from iToF, or for any other reason). The number of readout taps may be correspondingly higher than the number of outputs.

[0039] However, it is assumed that not all outputs are read by the readout circuit for different reasons. For example, the outputs may be defective such that they cannot be read. In the case of multiple imaging sub-elements that establish one imaging element (e.g., a macro pixel), it is not necessary to read all outputs. This may be, for example, in a low resolution mode or when the ToF circuit is driven in any other imaging mode.

[0040] On the other hand, even if not all outputs are to be read by a readout circuit, the number of readout taps can still be equal to or higher than the number of outputs for similar reasons given above, such as reading out multiple outputs with more than one readout tap, etc.

[0041] Depending on how the present disclosure is implemented, one skilled in the art may understand that "time shifted" may also refer to "phase shifted" in some embodiments. Whether a read tap is time shifted or phase shifted may depend on which circuit elements or signals are actually used.

[0042] In some embodiments, the imaging element includes multiple imaging sub-elements, as discussed herein.

[0043] For example, a certain number of image sub-elements may be connected so that they can be read out in accordance with the present disclosure. For example, four two-tap pixels (e.g., based on CAPD) may be used, with each tap corresponding to an output of a macro-pixel, although any number of outputs and image sub-elements may be used and the present disclosure is not limited in that respect.

[0044] In an exemplary embodiment, eight outputs may be used, which is further described with reference to FIG.

[0045] 1A shows a ToF circuit 1 according to the present disclosure, including an imager 2 and a readout circuit 10. The ToF circuit 1 is implemented as a spatially multiplexed N-tap sensor having N / 2 2-tap pixels with one damping node in each pixel.

[0046] In this embodiment, imaging element 2 includes four imaging sub-elements A, B, C and D which are all two-tap ToF pixels.

[0047] Each of the image subelements A to D includes a dump node DuA, DuB, DuC, and DuD and a collection node A1, A2, B1, B2, C1, C2, D1, and D2. The dump nodes DuA to DuD include a transistor 3 connected with its gate to a readout circuit. At another node, the transistor 3 is connected to a photodiode 4. Also, via node 5, the transistor 3 and the photodiode 4 are connected with transistors 6 and 7, whose gates are the respective collection nodes A1 to D2.

[0048] However, any of the transistors referred to herein are not limited to a particular type. For example, NMOS transistors can be used for the timing diagram of FIG. 1B. However, the transistors may be any type of transistor or may include a different type of doping, such as PMOS, in which case the polarity of the signals in FIG. 1B may be reversed.

[0049] Moreover, by their respective other nodes, the transistors 6 and 7 are connected to the gates of further transistors 8 and 9 which are biased to the supply voltage.

[0050] Therefore, when signals are applied to the damping nodes DuA to DuD, the 2-tap pixels A to D can be controlled so that the currents generated in the photodiodes 4 are drawn toward the respective damping nodes DuA to DuD. When signals are applied to the collection nodes A1 to D2, the currents in the photodiodes are drawn toward the respective collection nodes A1 to D2.

[0051] Signals are applied by readout circuitry 10 via respective signal lines 11 connected from collection nodes A1 to D2 or from dump nodes DuA to DuD, e.g., line A1 is connected to collection node A1, line DuB is connected to dump node DuB, etc.

[0052] The readout circuit 10 further comprises a signal generator MV (modulation clock generator and driver short) configured to generate the respective signals.

[0053] FIG. 1B shows timing 15 of signals applied via signal line 11 to each node (damping nodes DuA to DuD and collection nodes A1 to D2).

[0054] In general, if an imaging sub-element of imaging sub-elements A to D should not be read out, the voltage applied to the respective damping node DuA to DuD is high (or logical one) and if the respective imaging sub-element is to be read out, the voltage applied to the respective damping node DuA to DuD is low (or logical zero). As mentioned above, if different transistors are used, the signals can be changed or even reversed.

[0055] In this embodiment, the image pickup devices A to D are read out one after the other, and the collection nodes A1 to D2 are read out one after the other.

[0056] Thus, at a first time, when nothing is being read, the signals on the damping nodes DuA-DuD are high and the signals on the collection nodes A1-D2 are low. When reading is started, at the same time, the signal at the first damping node A1 is high, while the signal at the damping node DuA is set low.

[0057] After a predefined signal width, the signal on A1 is set low again, and immediately afterwards, the signal on A2 is set high for the predefined signal width. By again setting A2 low, DuA is set high again and DuB is set low. At the same time, B1 is set high for the predefined signal width. After B1 is set low, B2 is set high for the predefined signal width. Then, by again setting DuB high and DuC low, C1 and C2 are set to go high one after the other for the predefined signal width. After again setting C2 low, DuC is set high again and DuD is set low. As a result, D1 and D2 are set high one after the other for the predefined signal width. Then, DuD is set high again.

[0058] In this embodiment, all collection nodes A1-D2 are read out for the same predetermined time, but the readout time for each collection node may be adapted accordingly and may be different, for example, if different photodiodes (with different quantum yields) are used, or if the image sub-elements have different components, or based on calibration.

[0059] Furthermore, timing 15 shows only a portion of the overall modulation pattern and can be adapted or modified as appropriate: the pattern may be repeated within a column over the entire exposure when a micropixel (or image subelement) collects photoelectrons (i.e., when the respective dump node (or dump gate) is low).

[0060] The implementation shown in Fig. 1 uses the commonly known two-tap technology. Thus, known imaging sub-elements can be used and it may be sufficient to adapt only the read-out and read-out circuits. Furthermore, the imaging sub-elements may be based on mature CIS (CMOS image sensor) technology and the respective outputs (taps) may be provided close to each other, thus optimizing the modulation speed.

[0061] Furthermore, in some embodiments, mismatch / offset of each readout circuit can be mitigated if the readout circuit includes a shared source follower, which can be shared for all taps within one macro pixel (in this embodiment, an image sensor).

[0062] Furthermore, the present disclosure is not limited to any number of image sub-elements, such as the four image sub-elements of FIG.

[0063] In some embodiments, each sub-element includes at least one output. In the above example, each imaging element has two outputs, but different types of imaging elements having different types of outputs may be used in accordance with the present disclosure and the outputs may be controlled accordingly.

[0064] 2A shows an embodiment of a ToF circuit 20 including an imaging element 21 having nine imaging sub-elements 22, each having one output 23, implemented as a pulse generator. The nine imaging sub-elements 22 are arranged in a 3x3 pixel array. Each imaging sub-element further includes a SPAD 24 (single photon avalanche diode), and a counter 25.

[0065] The ToF circuit 20 further includes a ToF readout circuit 26 including a signal generation MV and signal lines 27. There are three of them, each of which is connected to an imaging subelement of each row of the pixel array. Thus, the signal line G[0:2] is connected to the output 23 of the first row, the signal line G[3:5] is connected to the output 23 of the second row, and the signal line G[6:8] is connected to the output 23 of the third row. Thus, via the line G[0:2], modulation signals G[0], G[1], and G[2] can be applied to the output of the first row, where the signal G[0] is applied to the first pixel of this row, G[1] is applied to the second pixel of this row, and G[2] is applied to the third pixel of this row. Correspondingly, modulation signals G[3], G[4], and G[5] can be applied to the first, second, and third pixels in the second row via signal lines G[3:5], and modulation signals G[6], G[7], and G[8] can be applied to the first, second, and third pixels in the third row via signal lines G[6:8].

[0066] When a modulated signal is applied to the respective pulse generator or output 23, the respective counter 25 is incremented if an avalanche signal is generated in the SPAD 24. In the case of mutually driven outputs 23, the counting results of the different counters 25 can be associated with respective time bins.

[0067] 2B shows the timing 28 of signals G[1] to G[8] being applied to output 23. Such an embodiment may also be referred to as spatial multiplexing.

[0068] As can be seen from timing 28, before the read phase begins, signals G[1] through G[8] are set low. Then signal G[0] is set high for a predetermined time and then set low again. After that, signal G[1] is set high for a predetermined time and then set low again. The same applies to the remaining signals G[2] through G[8], which are all applied immediately after each other.

[0069] Therefore, in this embodiment, a similar pattern to that of FIG. 1 is applied, thus realizing N-tap operation in a SPAD-based imager.

[0070] Spatial multiplexing using a mosaic has been described with respect to the exemplary embodiment of Figures 1 and 2. In such an embodiment, the layout of a macro-pixel may include a plurality of (sub)pixels in a mosaic-like manner, in which each of the (sub)pixels may individually correspond to at least one tap in an N-tap sensor and thus may operate in turn to resolve the incident light of the corresponding phase. The (sub)pixels may be based on active pixel devices or other devices, e.g., 3T, 4T, SPAD, APD, etc.

[0071] Mosaic can refer to the use of multiple pixels together, however, these do not necessarily have to be adjacent pixels, as any mosaic pattern can be envisaged depending on the circumstances.

[0072] If the tap is not addressed it will generate a signal, but such a signal may be arranged, for example, to 1000 kHz, ...

[0073] For example, different regions of the imager may have capacitance with respect to another region. For example, if the ToF circuitry is implemented in a semiconductor, different doped regions (as outputs) may be provided, in which case charge may be collected and read out via a readout tap, which may be applied to the different doped regions. The doped regions may constitute a parasitic capacitance with another layer of the semiconductor (e.g., a readout layer), and thus may be represented symbolically by a capacitor. However, depending on how the ToF circuitry is implemented, a "real" capacitor may be used as well.

[0074] In some embodiments, multiple time-shift read taps are applied to multiple capacitances based on a main read capacitance to which each time-shift read signal is applied and based on multiple sub-capacities that are each read with a respective one of the time-shift read signals.

[0075] To illustrate such an embodiment, reference is now made to FIG.

[0076] 3A shows two embodiments in which temporal multiplexing can be implemented.Temporal multiplexing can refer to the sharing of one imager by multiple readout nodes.

[0077] 3A shows a passive ToF (integrator) circuit 30. The ToF circuit 30 includes an imager 31 that includes a photodiode and an analog modulator 32, which further includes a storage element and charge circulation circuit (not shown).

[0078] Also provided is a read circuit 33 for reading out the analog modulator 32 with multiple inputs and multiple outputs.

[0079] The read circuit 33 is connected to the analog modulator 32 at the gate of a transistor 34. The transistor 34 is further connected to a supply voltage VDD and to a further transistor 35.

[0080] In addition, transistor 36 is connected to transistor 34 and to the gate of analog modulator 32 .

[0081] On the right side of FIG. 3A, a readout circuit 40 is shown that is implemented as a single-ended capacitive transimpedance based active integrator with N memory elements, which acts as both an integrating element and a circulating element.

[0082] The readout circuit 40 is connected to a capacitor 41 which connects the readout circuit 40 to an imager (not shown).

[0083] The readout circuit 40 includes an operational amplifier 42 which has one input node connected to a reference voltage V_ref and another input node connected to a capacitor 43 and a reset switch (RST) 44 which is then connected to the output of the operational amplifier 42.

[0084] Furthermore, the output of the operational amplifier 42 is connected to a switch SH, which, when closed, establishes a connection between the readout circuit 40 and the capacitor 41 .

[0085] The readout circuit 41 further includes five sub-circuits 45. It should be noted that the present disclosure is not limited to five sub-circuits 45, as this is merely an exemplary embodiment and any number of sub-circuits may be envisioned. It should also be noted that although five sub-circuits are shown, this embodiment includes eight sub-circuits, with five being shown for illustrative purposes only.

[0086] Each sub-circuit 45 includes a capacitor 46 that is connected to the capacitor 43 such that the sub-circuit 45 is coupled to the input of the operational amplifier 42 .

[0087] The sub-circuit 45 also has two parallel signal lines 47 and 48 (on the other side) connected to the capacitor 46 .

[0088] Signal line 47 includes switch GD and is connected to the output of operational amplifier 42. Switch GD is symbolic of a number of switches, the timing of which is further explained with reference to FIG. 3B and is listed as GD1 through GD8.

[0089] Signal line 48 includes a switch SWGND and is connected to a reference voltage V_ref. Switch SWGND is symbolic of a number of switches, the timing of which is further discussed with reference to FIG. 3B and listed as SWGND1 through SWGND8.

[0090] If the switch SH is closed, a read can be performed according to the following exemplary embodiment, which will be discussed with reference to FIG. 3B.

[0091] 3B shows a timing 50 in which the read circuit 40 may be driven. In the following, when a high signal (or a logical one) is applied, the respective switch of the read circuit 40 is closed, and when a low signal (or a logical zero) is applied, the respective switch of the read circuit 40 is opened.

[0092] In the reset phase 51, all nodes are reset to a reset voltage by a high signal 52 applied to switch RST 44 and GD1-GD7 (corresponding to switch GD in FIG. 3A, but enumerated to distinguish the subcircuits 45).

[0093] Furthermore, a high signal 53 is applied to switch GD8 (which in this embodiment is the main switch for the main capacitance), which extends longer than high signal 52 into the integration phase 55. During the integration phase, a modulation pattern similar to that discussed with reference to FIG. 2B is applied, i.e. after the mutual switches have been closed for a predefined time. However, contrary to FIG. 2B, switch GD8 is closed for a longer time than switches GD1-GD7, since GD8 is used as a dump node (or dump tap). The modulation pattern is then repeated until a predefined number of modulation cycles have been performed.

[0094] During the reset phase 51 and the integration phase 55, the signals of the switches SWGND1 to SWGND8 are kept low.

[0095] After the integrate phase 55, the dump node GD8 is reset by applying a high signal to RST 44 and keeping the signal high on GD8 during the dump node reset phase 56. The remaining signals are kept low, i.e. all other switches remain open.

[0096] After the dump node reset phase 56, RST 44 is set low again, but GD8 is kept high. This phase is also called the charge cycling phase. In accordance with the present disclosure, multiple charge cycling phases are performed in which each node is read out in turn.

[0097] In the first charge circulation phase 57, charge is transferred from GD7 to GD8 and is drawn from GD8 because GD8 is still high (i.e., closed). Additionally, SWGND7 is set high, but GD7 is set low, allowing charge to flow from GD7 to GD8.

[0098] After the first charge circulation phase 57, a second charge circulation phase 58 is performed, where charge flows from GD6 to GD7 by setting GD7 high and SWGND6 high. The same is true for GD5, its charge is transferred to GD6, and so on. After all switches (or tabs) have done one charge transfer, a new integration phase starts, and the modulation pattern on GD is also misaligned by one tab from the first integration period. The combination of integration and charge transfer period repeats N times (N is equal to the number of tabs excluding the dump node, i.e., 8 times in this embodiment) or N integrations.

[0099] Such charge cycling can be used to equalize transfer rate mismatches between the photodiodes and the different capacitors.

[0100] After this operation, starting with charge circulation 57, all tabs are read out; i.e., charge is transferred from GD7 to GD8 for readout. After the first charge circulation phase 57, a second charge circulation phase is performed in which charge flows from GD6 to GD7 by bringing GD7 high, SWGND6 high, etc. Then the first charge circulation phase 57 is performed again, and charge is read out from GD8. The same applies to GD5, whose charge is transferred in a similar manner to GD8, etc. Charge circulation as discussed above can be used to ensure that readout is performed on one tab, for example to remove offsets in the readout voltage due to capacitance mismatch.

[0101] In this embodiment, the capacitive transimpedance amplifier CTIA is used for both integration and charge circulation, but if used only for charge circulation, the reset timing may only be related to the charge circulation and readout phases.

[0102] In some embodiments, each capacitance is coupled to at least two read switches as discussed herein.

[0103] Temporal multiplexing has been described with reference to Fig. 3. In this non-limiting example, N-tap operation is performed by N analog integrators or N counter elements sharing the same pixel device. In such an embodiment, there can be no pixel mismatch since only one pixel device can be used for some taps.

[0104] In an analog implementation, the N memory elements may be realized by N integrated capacitors, and the switching operation between the capacitors may be realized by analog switches. In a counting implementation, the N memory elements may be realized by N counters, and the switching operation between the capacitors may be realized by multiplexers.

[0105] In some embodiments, the multiple outputs correspond to multiple counters, which in some embodiments represent multiple outputs or multiple taps.

[0106] For example, a first counter (as known from dToF) may count photons for a first time interval, a second counter may count photons for a second time interval, and so on.

[0107] The counters may be timed based on the corresponding circuitry described below with respect to FIG.

[0108] 4A shows a ToF circuit 60 that includes a SPAD 61 that transfers charge to a pulse generator 62. When the pulse generator detects an avalanche signal from the SPAD 61, a pulse is generated and sent to a multiplexer 63.

[0109] The control line 64 is connected to the multiplexer 63 such that each of the counters 65a, 65b, 65c, 65x (where x represents any number, i.e., the present disclosure is not limited to four counters as shown in FIG. 4A) is driven by the multiplexer 63 according to a modulated signal transmitted by the control line 64.

[0110] FIG. 4B shows a timing diagram 66 in which multiplexer 63 drives counter 66.

[0111] Similar to the modulation pattern of Figure 2B, different modulation signals a, b, c, and x (where x represents any number and the letters used herein correspond to the letters used for counters 65a, 65b, 65c, and 65x) are applied successively. The upper counter 65a is incremented when a pulse from pulse generator 62 occurs at the same time that the upper modulation signal is high. Thus, the second counter from above 65b is incremented when a pulse from pulse generator 62 occurs at the same time that the second modulation signal from above is high, and so on.

[0112] Thus, in accordance with timing diagram 66 of FIG. 4B, the multiplexer control signals are generated in a manner that time multiplexes the outputs of image SPAD 61 onto corresponding taps.

[0113] The multiplexer is not limited to any particular element and may be implemented based on NAND gates, AND gates, transmission switches, and the like.

[0114] Based on the counter, as is commonly known, a histogram 67 can be generated based on which depth generation can be determined, as shown in FIG. 4C.

[0115] Thus, in some embodiments, as discussed herein, a histogram is generated based on counts in multiple counters.

[0116] A non-limiting example of the DEM (Dynamic Element Matching) technique was discussed with reference to FIG. 4. For example, the mismatch cancellation technique by tap circuit / charge cycling can be performed by an analog integrator. In such an embodiment, the N-tap may be implemented by an N-tap pixel concept such as a gate pixel (having N gates) or by another pixel modulated with an analog integrator. To reduce the offset, charge can be cycled through the analog memory component of the integrator while the laser phase can be changed accordingly.

[0117] In some embodiments, the counter is a bidirectional counter.

[0118] When a bidirectional counter is used, the histogram may be less complex and therefore less silicon space may be occupied by the counter. Thus, in some embodiments, the histogram is generated based on the counts in the bidirectional counter, with the entries of the histogram corresponding to the relative counts with respect to their respective neighboring entries.

[0119] Referring to FIG. 5, a ToF circuit with a bidirectional counter and its timing is described.

[0120] In FIG. 5A, a ToF circuit 70 is shown including a SPAD 71 connected to a pulse generator 72. In response to a signal from the SPAD 71, the pulse generator 72 is configured to generate a pulse that is sent to a multiplexer 73. The multiplexer is controlled by a ToF readout circuit according to the present disclosure such that the pulse is sent only to a predetermined counter according to a modulation signal or readout tap, as discussed herein. Thus, the ToF circuit further includes a number of bidirectional counters 74. In contrast to the counter 65 of FIG. 4A, the bidirectional counters 74 of this embodiment occupy less silicon space since they can perform delta counting and therefore need to conserve fewer count values.

[0121] To implement the delta counting, the bidirectional counter 74 is signed and operates cyclically.

[0122] For the nth tap, the corresponding counter n is configured to accumulate counts from the SPAD 71 in the positive or negative direction, while counter n-1 counts in the opposite direction. Thus, one counter is configured to process the differential signal value and partial ambient light count value of two adjacent taps.

[0123] Additionally, adders 82 are provided, each connecting one output of the multiplexer 75 to two adjacent counters (instead of one counter as in FIG. 4A). The adders may be any type of adder, such as a digital OR gate.

[0124] Figure 5B shows the timing 75 of multiple taps applied to multiplexer 73. The taps 75 are based on time-shifted modulated signals applied to different switches 76 in multiplexer 73 one after the other, similar to Figure 2B, such that a pulse from pulse generator 72 is sent to a respective counter 74 connected to each switch 76.

[0125] 5C shows a timing signal 77 that the counter 74 is configured to count. This timing signal 77 includes multiple time-shifted modulation signals 78 that are applied after each other. The time-shifted modulation signals 78 are reused for the taps 75 (or multiplexer control signals).

[0126] Counter 74 is configured to count up (i.e., plus one, without limiting this disclosure in that respect) when high region 79 of modulated signal 78 for the respective counter overlaps with a pulse of pulse generator 72. Counter 75 is configured to count down (i.e., minus one, without limiting this disclosure in that respect) when low count region 80 of modulated signal 78 overlaps with a pulse. Modulated signal 78 further includes a region 81 where no counting takes place (also referred to as a "marker-free region"). Region 81 overlaps with a high region of the modulated signal of an adjacent counter, thereby allowing delta counting to take place.

[0127] 5D shows a delta histogram 85 (right side) generated based on the ToF circuit 70. On the left side, the histogram 67 of FIG. 4D is shown to compare the two histograms. In addition, a differential value bar 86 between the first and second bars is shown on the second bar from the left of the histogram 67, which represents the differential value between the first and second bars of the histogram 67.

[0128] Since the second bar is lower than the first bar, the derivative value is negative. This derivative value is therefore depicted as a negative second value in the delta histogram 85 on the right. The first bar of the histogram 85 is derived in a similar manner, based on a comparison of the first bar with the last bar. It should be noted that, according to the embodiment of FIG. 5, it is not necessary to generate the histogram 67, based on which the delta histogram 85 is generated, since the ToF circuit 70 is configured to generate the delta histogram 85 directly.

[0129] Two peaks are shown in delta histogram 85. Because delta counts are performed, at least two peaks are seen every time a ToF measurement is performed, where the two peaks are signed, i.e., one is positive and the other is negative.

[0130] Based on the delta histogram 85, the histogram 67 can be reconstructed without the common mode count. Alternatively, based on the two peaks, the location of the peak can be concluded directly based on the flip of the large delta count value with different signs between the two counters. The remaining bins are small values ​​(i.e., below a predefined threshold) because the common mode counts may have already been subtracted or not counted in the first place. In the above embodiment, the peak is located in the sixth bin looking from the left.

[0131] A non-limiting example of a delta counting circuit and method has been described with reference to Figure 5. In such a configuration, a smaller size macrocell may be required compared to the embodiment of Figure 4 because fewer unit cells or counting elements may be required due to the smaller size of the histogram. Thus, the embodiment of Figure 5 may be more area efficient.

[0132] In general, delta counting can refer to storing the difference between two adjacent taps instead of storing the raw count value. As mentioned above, signed bidirectional counters can be used without limiting the disclosure in that respect. All counters can operate sequentially, but two adjacent counters can be enabled simultaneously while one of the counters can count up and the other can count down.

[0133] The counter may be adapted to process the differential signal value of two adjacent taps and the clipped ambient light value (i.e. the ambient light value may not be taken into account as it may be immediately differenced during exposure by delta counting), so that in total the counter area is conserved in contrast to a counter adapted only to store the received light signal and ambient light during the entire exposure time.

[0134] Some embodiments relate to a time-of-flight readout method comprising applying a plurality of time-shifted readout taps to a plurality of outputs of an imager, the time-shifted readout taps being shifted in time with respect to each other such that the plurality of outputs of the imager are read out one after the other.

[0135] The ToF readout method may be implemented in a ToF circuit and / or a ToF readout circuit according to the present disclosure.

[0136] In some embodiments, the imaging element includes a plurality of imaging sub-elements, as discussed herein. In some embodiments, each sub-element includes at least one output, as discussed herein. In some embodiments, the plurality of outputs corresponds to a plurality of capacitances of the imaging element, as discussed herein. In some embodiments, the plurality of time-shift readout taps are applied to a plurality of capacitances based on a main readout capacitance, which is read out for each time-shift readout signal, as discussed herein, based on a plurality of secondary capacitances read out with a time-shift readout signal of one of the plurality of time-shift readout signals. In some embodiments, each capacitance is coupled to at least two readout switches, as discussed herein. In some embodiments, the plurality of outputs corresponds to a plurality of counters, as discussed herein. In some embodiments, the histogram is generated based on counts in a plurality of counters, as discussed herein. In some embodiments, the counter is a bidirectional counter, as discussed herein. In some embodiments, the histogram is generated based on counts in a bidirectional counter, where entries of the histogram correspond to relative counts with respect to their respective neighboring entries, as discussed herein.

[0137] The methods described herein are also implemented in some embodiments as a computer program executed on a computer and / or processor that causes the computer and / or processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided that stores a computer program product that, when executed by the above-mentioned processor, causes the method described herein to be performed.

[0138] FIG. 6 illustrates a ToF readout method 90 according to the present disclosure, where multiple readout taps are applied to multiple image sub-elements.

[0139] Thus, at 91, multiple time-shifted readout taps are applied to multiple image sub-elements as discussed herein. The image sub-elements are based on CAPDs, each having multiple outputs.

[0140] FIG. 7 shows a further embodiment of a ToF readout method 95 according to the present disclosure, in which multiple time-shifted readout taps are applied to multiple capacitances of the imager.

[0141] Thus, at 92, as in the embodiment of FIG. 3, multiple time-shifted read taps are applied to multiple capacitances as discussed herein.

[0142] FIG. 8 depicts a further embodiment of the ToF readout method 100 according to the present disclosure in which multiple time-shifted readout taps are applied to multiple counters.

[0143] Thus, at 101, a number of time-shifted read taps are applied to a number of counters, which in this embodiment are bidirectional counters as discussed with reference to FIG.

[0144] FIG. 9 illustrates a further embodiment of a ToF readout method 105 according to the present disclosure, in which a histogram is generated based on multiple counters.

[0145] At 106, multiple time-shift readout taps are applied to multiple bidirectional counters as discussed herein.

[0146] At 107, a delta histogram is generated as discussed herein.

[0147] At a high level, FIG. 10 illustrates one embodiment of a time-of-flight imaging system 110, here embodied as a ToF camera, that can be used to provide depth sensing or distance measurement, configured to perform methods as discussed herein, and including time-of-flight optical circuitry 117 that provides control of the ToF device 110 (and corresponding processor, memory, and storage, not shown, but as would be commonly known to one of ordinary skill in the art).

[0148] The ToF imaging system 110 comprises a pulsed light source 111, which includes a light emitting element (based on a laser diode), which in this embodiment is a narrowband laser element.

[0149] A light source 111 emits pulses of light onto a scene 112 (an area or object of interest) which reflects the light. By repeatedly emitting light onto the scene 112, the scene 112 can be scanned, as is commonly known to those skilled in the art. The reflected light is focused by an optical stack 113 onto a photodetector 114.

[0150] The time-of-flight light circuit 117 also configures control of the light source, including corresponding control circuitry (not shown).

[0151] The light detection unit 114 includes an image sensor 115 realized based on a number of SPADs (single photon avalanche diodes) formed in an array of pixels (imaging elements), and a microlens array 116 that focuses light reflected from the scene 112 onto the image sensor 115 (for each pixel of the image sensor 115).

[0152] The emission time information is provided from the light source 111 to a time-of-flight optical circuit 117 which includes a time-of-flight readout circuit 118 which also receives respective time information from the image sensor 115 when light reflected from the scene 112 is detected. Based on the emission time information received from the light source 111 and the time-of-arrival information received from the image sensor 115, the time-of-flight readout circuit 118 calculates the round trip time of the light emitted from the light source 111 and reflected by the scene 112, and based thereon calculates the distance d (depth information) between the image sensor 115 and the scene 112.

[0153] The depth information is provided to a 3D image reconstruction unit 119 of the time-of-flight optical circuit 117 which reconstructs (generates) a 3D image of the scene 112 based on the depth information received from the time-of-flight measurement device 118 .

[0154] It should be noted that the division of ToF circuitry 117 into units 118 and 119 is for illustrative purposes only, and the present disclosure is not limited to any particular division of functionality in any particular unit. For example, ToF circuitry 117 may be implemented by a respective programmed processor, a field programmable gate array, or the like.

[0155] The methods described herein may also be implemented as a computer program that, when executed on a computer and / or processor, causes the computer and / or processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided that stores a computer program product that, when executed by a processor, such as the processors described above, causes the computer and / or processor to perform the method.

[0156] All units and entities described in this specification and claimed in the accompanying claims may be implemented as integrated circuit logic, for example on a chip, unless otherwise stated, and the functions provided by such units and entities may be implemented by software, unless otherwise stated.

[0157] In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be understood that computer programs providing such software control, and the transmission, storage, or other media on which such computer programs are provided, are contemplated as aspects of the present disclosure.

[0158] The present technology can also be configured as follows. (1) an imaging element; a readout circuit configured to apply a plurality of time-shifted readout taps to a plurality of outputs of the imager; Equipped with The time-shifted readout taps are time-shifted relative to one another such that the outputs of the imager are read out one after the other. Time-of-flight circuit. (2) The imaging element includes a plurality of imaging sub-elements. A time-of-flight circuit as described in (1). (3) Each subelement contains at least one output. (2) A time-of-flight circuit as described above. (4) The plurality of outputs correspond to a plurality of capacitances of the imaging element. A time-of-flight circuit as described in (1). (5) the plurality of time-shifted read taps are applied to the plurality of capacitances based on a main read capacitance to which each time-shifted read signal is applied, and based on a plurality of sub-capacities each read by a respective one of the plurality of time-shifted read signals; (4) A time-of-flight circuit as described above. (6) Each capacitor is coupled to at least two read switches. A time-of-flight circuit according to (4) or (5). (7) the plurality of outputs correspond to a plurality of counters; A time-of-flight circuit as described in (1). (8) A histogram is generated based on the counts in the plurality of counters. (7) A time-of-flight circuit as described in (7). (9) The counter is a bidirectional counter. A time-of-flight circuit according to (7) or (8). (10) the histogram is generated based on counts in the bidirectional counter; The entries of the histogram correspond to relative counts for their respective neighboring entries. A time-of-flight circuit as described in (9). (11) applying a plurality of time-shifted readout taps to a plurality of outputs of an imager; The time-shifted readout taps are time-shifted relative to one another such that the outputs of the imager are read out one after the other. Flight time readout method. (12) The imaging element includes a plurality of imaging sub-elements. (11) Flight time readout method. (13) Each subelement contains at least one output. A method for reading out a time of flight according to (12). (14) The plurality of outputs correspond to a plurality of capacitances of the imaging element. A method for reading out a time of flight according to (11). (15) The plurality of time-shifted read taps are applied to the plurality of capacitances based on a main read capacitance to which each time-shifted read signal is applied, and based on a plurality of sub-capacities each read by a respective one of the plurality of time-shifted read signals. A method for reading out a time of flight according to (14). (16) Each capacitance is coupled to at least two read switches. A method for reading out a time of flight according to (14) or (15). (17) The plurality of outputs correspond to a plurality of counters. A method for reading out a time of flight according to (11). (18) A histogram is generated based on the counts in the plurality of counters. A method for reading out a time of flight according to (17). (19) The counter is a bidirectional counter. A time-of-flight readout method according to (17) or (18). (20) the histogram is generated based on counts in the bidirectional counter; The entries of the histogram correspond to relative counts for their respective neighboring entries. A method for reading out a time of flight according to (19). (21) A computer program comprising a program code which, when executed on a computer, causes the computer to execute the method according to any one of (11) to (20). (22) A non-transitory computer-readable recording medium storing a computer program product that, when executed by a processor, causes the computer to perform any one of the methods described in (11) to (20).

Claims

1. An imaging element; a readout circuit configured to apply a plurality of time-shifted readout taps to a plurality of outputs of the imager; Equipped with The time-shifted readout taps are time-shifted relative to one another such that the outputs of the imager are read out one after the other. Time-of-flight circuit.

2. The imaging device includes a plurality of imaging sub-elements.

2. The time-of-flight circuit of claim 1.

3. Each subelement contains at least one output 3. The time-of-flight circuit of claim 2.

4. The plurality of outputs correspond to the plurality of capacitances of the imaging element.

2. The time-of-flight circuit of claim 1.

5. The plurality of time-shifted read taps are applied to the plurality of capacitances based on a main read capacitance to which each time-shifted read signal is applied, and based on a plurality of sub-capacities each read by a respective one of the plurality of time-shifted read signals.

5. The time-of-flight circuit of claim 4.

6. Each capacitance is coupled to at least two read switches.

5. The time-of-flight circuit of claim 4.

7. The plurality of outputs corresponds to a plurality of counters 2. The time-of-flight circuit of claim 1.

8. A histogram is generated based on the counts in the plurality of counters.

8. The time-of-flight circuit of claim 7.

9. The counter is a bidirectional counter.

8. The time-of-flight circuit of claim 7.

10. The histogram is generated based on counts in the bidirectional counter; The entries of the histogram correspond to relative counts for their respective neighboring entries.

10. The time-of-flight circuit of claim 9.

11. applying a plurality of time-shifted readout taps to a plurality of outputs of an imager; The time-shifted readout taps are time-shifted relative to one another such that the outputs of the imager are read out one after the other. Flight time readout method.

12. The imaging device includes a plurality of imaging sub-elements. A method for time-of-flight readout according to claim 11.

13. Each subelement contains at least one output A method for time-of-flight readout according to claim 12.

14. The plurality of outputs correspond to the plurality of capacitances of the imaging element. A method for time-of-flight readout according to claim 11.

15. The plurality of time-shifted read taps are applied to the plurality of capacitances based on a main read capacitance to which each time-shifted read signal is applied, and based on a plurality of sub-capacities each read by a respective one of the plurality of time-shifted read signals.

15. A method for time-of-flight readout according to claim 14.

16. Each capacitance is coupled to at least two read switches.

15. A method for time-of-flight readout according to claim 14.

17. The plurality of outputs corresponds to a plurality of counters A method for time-of-flight readout according to claim 11.

18. A histogram is generated based on the counts in the plurality of counters.

18. A method for time-of-flight readout according to claim 17.

19. The counter is a bidirectional counter.

18. A method for time-of-flight readout according to claim 17.

20. The histogram is generated based on counts in the bidirectional counter; The entries of the histogram correspond to relative counts for their respective neighboring entries.

20. A method for time-of-flight readout according to claim 19.

Citation Information

Patent Citations

  • Method and computer program for operating a photodetector

    DE102014207599A1

  • Distance measuring device, and driving method of distance measuring device

    JP2015215181A

  • Distance sensor

    JP2017191042A

  • Linear-mode avalanche photodiodes without excess noise.

    JP2021522694A

  • Distance image measurement device, distance image measurement system, and distance image measurement method

    WO2020149243A1