Time-of-flight circuit and time-of-flight method

Abstract

The present disclosure generally relates to a time-of-flight circuit configured to apply a set of detection time intervals to at least one light detection event to determine a time point of the at least one light detection event, wherein the set of detection time intervals has a predetermined detection pattern encoding a predetermined time point.

Description

Technical Field

[0001] This disclosure generally relates to time-of-flight circuits and time-of-flight methods. Background Technology

[0002] Typically, time-of-flight (ToF) systems used to measure distances to a scene are known. Direct time-of-flight (dToF) and indirect time-of-flight (iToF) are different. In the case of dToF, the round-trip delay of the emitted light is measured, and the distance is derived "directly" from the round-trip delay. Typically, dToF sensors can be based on technologies such as SPAD (single-photon avalanche diode).

[0003] In the case of iToF, which can typically be based on technologies such as CAPD (current-assisted photonic demodulator), the phase shift of the emitted light is determined by sampling the gates (or multiple transistor gates) of transistors included in or coupled to the CAPD.

[0004] In the case of iToF, a light pulse, such as a square light pulse of a predetermined frequency, can be emitted by a light source. This light pulse is reflected from the scene (e.g., an object) and received by the CAPD.

[0005] CAPD is typically configured to mix the generated signal (e.g., photocurrent) with a demodulation frequency that may have approximately the same frequency as the optical pulse but with different phase delays, such as 0°, 90°, 180°, and 270°, thereby allowing the ToF phase delay to be estimated, for example, by using the ARCTAN function.

[0006] This measurement can be repeated a second time (or a third or fourth time, etc.) at a second demodulation frequency to reduce measurement uncertainty and obtain a clear range (i.e., distance).

[0007] Each measurement may be referred to as a microframe, and the microframe measurement may be repeated several times (e.g., eight times), where each measurement may be combined to determine the distance, where certain parameters are assumed to be constant, such as the reflectivity of the scene, the distance to the scene, and the background lighting (i.e., ambient light).

[0008] Despite the existence of technologies that provide time-of-flight measurements, there is still a desire to provide time-of-flight circuitry and methods. Summary of the Invention

[0009] According to a first aspect, this disclosure provides a time-of-flight circuit configured to apply a detection time interval group to at least one optical detection event to determine the time point of at least one optical detection event, wherein the detection time interval group has a predetermined detection pattern that encodes the predetermined time point.

[0010] According to a second aspect, this disclosure provides a time-of-flight method comprising: applying a group of detection time intervals to at least one optical detection event to determine a time point of at least one optical detection event, wherein the group of detection time intervals has a predetermined detection pattern that encodes a predetermined time point.

[0011] Other aspects are set forth in the dependent claims, the following description, and the accompanying drawings. Attached Figure Description

[0012] The embodiments are described by way of example with reference to the accompanying drawings, in which:

[0013] Figure 1 A timing diagram according to this disclosure is depicted;

[0014] Figure 2 A timing diagram for driving a light source according to the present disclosure is shown;

[0015] Figure 3 A block diagram illustrating the configuration of an image sensor based on CAPD technology is shown.

[0016] Figure 4 A block diagram of a readout circuit for sequential readout of pixels including SPADs is depicted;

[0017] Figure 5 A block diagram of a readout circuit for parallel readout of pixels including SPADs is shown;

[0018] Figure 6 A block diagram for driving a Gray code counter is depicted;

[0019] Figure 7 A block diagram of another Gray code counter is depicted;

[0020] Figure 8 A block diagram depicting another embodiment of a Gray code counter;

[0021] Figure 9 An exemplary data compression diagram according to this disclosure is shown;

[0022] Figure 10 A block diagram depicting the method according to this disclosure is provided;

[0023] Figure 11 A block diagram depicting another method according to this disclosure;

[0024] Figure 12 A block diagram of the first alternative method is depicted;

[0025] Figure 13 A block diagram of the second alternative method is depicted;

[0026] Figure 14A block diagram depicting another method according to this disclosure;

[0027] Figure 15 A block diagram of a SPAD-based ToF camera is depicted; and

[0028] Figure 16 A block diagram of a CAPD-based ToF camera is depicted. Detailed Implementation

[0029] Provide a reference Figure 1 Before providing a detailed description of the embodiments, a general description will be given.

[0030] As mentioned earlier, in the case of dToF based on SPAD technology, a detection histogram is typically generated that correlates the detection time (i.e., flight time) with the corresponding count value of the detection time.

[0031] However, it has been recognized that this will use a certain amount of memory and there is generally a desire to reduce the amount of memory used in time-of-flight systems, for example, to increase the number of measurements to be performed, to reduce system size, etc.

[0032] On the other hand, iToF can be based on CAPD technology, and iToF measurements may be noisy and / or associated with low confidence levels, which may degrade the measurement.

[0033] However, it is desirable to increase the signal-to-noise ratio and to increase the confidence level used to improve distance determination.

[0034] In addition, it is necessary to compensate for shot noise from background light (i.e., ambient light), nonlinearity in distance determination, multipath propagation of emitted light, and loss of precision and accuracy.

[0035] For example, it is well known that the heat generated by a light source (such as a laser) can affect the light output power, which can affect the measurement accuracy, even though it can be assumed that the brightness level (output power) is constant.

[0036] In addition, for example, the concave corners of a room or object and the so-called flying pixels at the edges of objects can cause the mixed signal to become blurred, thus reducing accuracy.

[0037] Since the ToF measurement principle may be based on analog measurement data, nonlinearity may reduce accuracy. In addition, analog measurements require a large number of effective bits in the ADC (analog-to-digital converter) of each column (or row) of the image sensor.

[0038] Furthermore, the optical detection signal (e.g., photocurrent) can be modulated with a 50% duty cycle, which requires the light source to provide a corresponding light emission scheme, since applying a current or voltage pulse with a 50% duty cycle to, for example, a laser will typically cause the laser's duty cycle to shorten (e.g., 40%).

[0039] In addition, known systems require a certain amount of processing power to evaluate ToF measurements in real time because filtering may need to be performed between subsequent measurements and / or between adjacent pixels in order to provide reliable depth information, for example at the integer or floating-point level, which may also lead to high power consumption.

[0040] Therefore, some embodiments relate to time-of-flight circuits configured to apply a group of detection time intervals to at least one optical detection event to determine the time point of at least one optical detection event, wherein the group of detection time intervals has a predetermined detection pattern that encodes predetermined time points.

[0041] The time-of-flight circuitry can be or includes a processor, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), multiple coupled CPUs and / or GPUs, etc. The circuitry can be or includes an FPGA (Field-Programmable Gate Array) or any other integrated circuit (IC), included in or coupled to a time-of-flight image sensor (e.g., a pixel). The ToF circuitry can also be part of a ToF camera or a system that includes a ToF sensor and illumination equipment (including light sources, etc.).

[0042] The time-of-flight image sensor may be based on one or more CAPDs (current-assisted photodiodes) (e.g., in the case of iToF), one or more SPADs (single-photon avalanche diodes) (e.g., in the case of dToF), and may be based on CCD (charge-coupled device) technology, CMOS (complementary metal-oxide-semiconductor) technology, etc., but this disclosure is not limited to these specific examples.

[0043] Time-of-flight image sensors can be or include a single image sensor (e.g., a single pixel) or multiple (i.e., at least two) pixels arranged in an array, as is well known.

[0044] The time-of-flight circuit can be configured to apply a group of detection time intervals to at least one detection event.

[0045] It can be used as a control command, voltage signal, current signal, digital signal, analog signal, etc. (in one or more parts of a ToF image sensor).

[0046] The detection time interval group may include any number of detection time intervals equal to or greater than 1, so that ToF measurements can be performed efficiently in terms of time (e.g., if the number is 1 or less than or equal to a first predetermined threshold) or efficiently in terms of resolution (e.g., if the number is equal to or greater than a second predetermined threshold of multiple time intervals).

[0047] It should be noted that the first and second predetermined thresholds can also correspond to each other.

[0048] The detection time interval can be one or more time intervals for performing detection. The detection time interval can be based on conditions such as optical demodulation frequency, transmission gate demodulation frequency, distance from the scene (e.g., object), environmental factors (e.g., ambient light, lighting conditions), etc.

[0049] For example, the start and end of the detection time interval can be based on the distance to the scene, so that the expected return of reflected light can be covered within the detection time interval.

[0050] However, in some embodiments, the start and end of the detection time interval can be based on distance, so that the expected return may not be overwritten.

[0051] Due to this timing of the detection time interval, round-trip time encoding can be performed, as described below.

[0052] As mentioned earlier, optical detection events can be or are based on the detection of emitted light returning. Optical detection events can be or are based on the incidence of light on the ToF image sensor, the time point (or time interval) at which a current is generated in response to the incidence of light, the start of demodulation, the start of counting, etc.

[0053] The timing of at least one optical detection event can be determined by detecting current signals, voltage signals, power signals, etc., generated in response to at least one optical detection event.

[0054] The generated signal (current, voltage, power, etc.) can be detected in a predetermined detection mode by means of transistor sampling and / or driving, timing of register circuits, etc., and this detection mode can be configured or preset by the program based on conditions (e.g., illumination conditions, demodulation frequency, etc. as described above).

[0055] The predetermined detection mode may include multiple detection time intervals, which may be applied to the generated signal continuously, sequentially, simultaneously (partially) overlapping, etc., so that the generated signal can be detected in a subgroup of detection time intervals, rather than in another subgroup of detection time intervals.

[0056] Furthermore, multiple detection time intervals may include detection time intervals that differ in at least one aspect of phase and length, such that a predetermined detection pattern can constitute a unique encoding scheme, and that detection event time points can be assigned to subgroups of detection time intervals.

[0057] Encoding schemes that can be represented by binary code, hexadecimal code, etc., can encode predetermined time points so that the code indicates a combination of detection time intervals.

[0058] For example, in the case of binary code, each binary digit can indicate whether a detection event occurred within the detection time interval indicated by the corresponding binary digit.

[0059] For example, if there are two detection time intervals within a total detection time of 10 microseconds, the first detection time interval can cover the first five microseconds, and the second detection time interval can cover the last five microseconds. If the detection event occurs within the first five microseconds, the binary code can be 10, because a positive identification of the detection event can be assigned to binary 1, and a negative identification of the detection event can be assigned to binary 0. This disclosure is not limited to this.

[0060] In some embodiments, the predetermined detection pattern is based on Gray code.

[0061] Therefore, the detection time interval can be selected in such a way that its continuous (or simultaneous, or (partially) overlapping) application is Gray coding.

[0062] Furthermore, the generated code (as described above) may be the well-known Gray code.

[0063] In some embodiments, the time-of-flight circuit is further configured to apply the detection time interval group to a photon counter in a predetermined detection mode, the photon counter being coupled to a single photon avalanche diode.

[0064] A photon counter can be configured to determine, for example, the number of photons incident on a SPAD by measuring the photocurrent and / or voltage generated in the SPAD, directly or indirectly.

[0065] In some embodiments, events based on numerical results are detected within the SPAD.

[0066] One or more demodulation functions representing a predetermined detection mode can be used to compare multiple digital events when the sign of the demodulation function is positive with multiple events when the sign of the demodulation function is negative.

[0067] If the number of events with a positive sign in the demodulation function is greater than the number of events with a negative sign in the demodulation function, the bit can be encoded as 1. However, when the number of events with a negative sign in the demodulation function is greater than the number of events with a positive sign in the demodulation function, the encoded bit can be encoded as zero.

[0068] As mentioned above, symbol comparisons can be performed in parallel or sequentially.

[0069] Alternatively, comparison can be achieved using two separate photon counters (or event counters), one of which counts events with a positive sign and the other counts events with a negative sign.

[0070] The comparison can be implemented using an up / down counter, which increments the count when the sign is positive and decrements the count when the sign is negative (and vice versa).

[0071] Alternatively, a temperature shift register can be used for comparison, where the edge of the shift register shifts in the first direction when a positive sign is detected, and shifts in the opposite direction when a negative sign is detected.

[0072] As discussed in this paper, SPADs can be used in the case of dToF, which may include an active lighting system for determining the distance between the image sensor (including one or more SPADs) and the scene by measuring the round-trip delay required for light pulses to reach the object and return to the sensor.

[0073] To assess the distance to the scene, the dToF system can record a histogram of photons detected over a period of time (i.e., the detection time), which can be based on the maximum detection distance.

[0074] Histograms can be discretized in time according to an internal reference clock (e.g., a time-to-digital converter (TDC)), and this disclosure is not limited thereto (where the TDC itself may not be required for the grayscale encoding discussed herein).

[0075] To compensate for noise (such as noise caused by ambient light or internal system noise), multiple dToF measurements can be performed continuously (or sequentially) and added to a histogram so that a cumulative histogram can be generated for all measurements or subgroups of all measurements.

[0076] However, as mentioned above, cumulative histograms may require a certain amount of memory per pixel.

[0077] Histograms can be evaluated based on encoding schemes (such as the Gray code scheme discussed in this paper), and the encoded results can be stored in one or more memory nodes for each Gray code pattern, which can then be evaluated sequentially or in parallel, as will be discussed further below.

[0078] To determine the Gray code value (i.e., Gray code bit), the signal can be stored in two unsigned counters, which can be compared, and the larger value can be used to establish the Gray code value.

[0079] Gray code values ​​can also be determined using a single memory storage node for signed integers, and addition or subtraction can be performed on the values ​​based on the Gray code.

[0080] Gray code values ​​can also be determined by processing the cumulative histogram using a bidirectional shift register.

[0081] The detection time interval can be applied to a photon counter, allowing the photon counter to count whether a photon is detected within the detection time interval.

[0082] In some embodiments, multiple photon counters driven simultaneously are applied, so that the detection time interval is also applied simultaneously, and encoding can be performed efficiently.

[0083] In some embodiments, the photon counter is a Gray code counter, meaning that the timing of the controller and the determination of whether a photon is detected can be represented by Gray code.

[0084] In some embodiments, as described above, the time-of-flight circuitry is also configured to decode time points based on Gray code information generated in a Gray code counter that indicates the distance to the scene.

[0085] In some embodiments, the time-of-flight circuit is further configured to use a demodulation mode group demodulation current to assist the photon demodulator, thereby applying the detection time interval group.

[0086] These embodiments can be applied when the ToF image sensor includes a CAPD.

[0087] However, this disclosure is not limited to the case of CAPD, as it is generally applicable to photon demodulators as well.

[0088] Typically, in a CAPD, charge can be collected by accumulation in two (or more) detectors, where the charge is transferred to either the first or second detector depending on the sign of the demodulation function.

[0089] For example, if the sign of the demodulation function is positive, the charge is transferred to the first detector, and if the sign is negative, the charge is transferred to the second detector, so that the total charge accumulated in the first detector can be compared with the total charge accumulated in the second detector. Thus, if the total charge in the first detector is greater than the total charge in the second detector, it can be encoded as 1, and if the total charge in the second detector is greater than the total charge in the first detector, it can be encoded as 2, and vice versa.

[0090] Each demodulation mode can be included in a microframe (as described above), and a total of n microframes can be recorded, where the system period of time T can be repeated m times for each microframe.

[0091] During each microframe, minority carriers can be generated in response to the received light (e.g., based on photocurrent), which can interact with the demodulated signal f from each microframe. n The demodulated signal f in the sum i mix.

[0092] Each demodulated signal f i It can have a period T that includes a specific phase.i (i.e., the detection time interval). The period T can be selected. i This allows them to form Gray code relationships with each other.

[0093] Then, by combining the photocurrent signal with each demodulated signal f i The resulting mixed signal, generated by mixing (e.g., multiplication), can have a sign (positive or negative) indicating whether it corresponds to a one (plus sign) or a zero (minus sign) in Gray code, thereby generating a Gray code bit value for the distance to the scene.

[0094] For example, with 8 demodulated signals, i.e., n equals 8, a distance resolution of 256 bits can be achieved, resulting in a deviation of less than 0.5% compared to the specified distance.

[0095] Then, the digital mapping can provide the possibility of converting the generated Gray code into a standard digital code, where it is also conceivable to evaluate the direct digital results of SPAD.

[0096] In addition, the mixed signal may have a norm, such as the average of the peaks, which indicates the certainty of whether the recorded bits are correct.

[0097] On the other hand, an analog value with its sign (i.e., Gray code bits) can be recorded after each demodulated microframe. The norm of the signal may give the confidence level of the ToF measurement, which may indicate the quality of the measurement.

[0098] If a simulated value has a norm below a predetermined threshold compared to other simulated values ​​from the same measurement, the situation may be that the distance to the scene lies at the boundary between two Gray codes, which can further refine the depth. Overall, this may improve the accuracy to more than n bits.

[0099] In addition, recording analog values ​​can compensate for offsets caused by voltage followers and current sources caused by readouts.

[0100] Furthermore, it is conceivable to provide a low-offset comparator in each pixel (e.g., an optimized sense amplifier that may be synchronous or asynchronous) so that Gray code bits can be generated for each microframe immediately.

[0101] As is well known, errors can occur because each measurement has a certain probability of error.

[0102] For example, if the error rate is 10 -4 Furthermore, if 10 bits are measured (i.e., n=10), then in a complete frame comprising 10 microframes, there is approximately one in a thousand possible error bits. Thus, applying the time-of-flight circuit according to this disclosure can provide more accurate measurements than known systems.

[0103] Furthermore, the nonlinearity may (approximately) have no impact on the coding scheme of this disclosure.

[0104] When the received optical pulse is centered around the edge of the demodulation function, the statistical extension of the result of this one bit can be used to refine the distance estimate to more than n bits of precision, as described above.

[0105] Because Gray code can be used, only one Gray code bit can be located at the edge of the demodulation function at a time.

[0106] Furthermore, error bits can be picked out by considering consecutive frames, where the behavior of each bit in subsequent frames can be checked, and error bits can also be picked out by considering confidence levels (as described above).

[0107] In some embodiments, the sign of the average value of the mixed signal can be evaluated, wherein, in some embodiments, the sign can be based on a direct digital result of a ToF measurement.

[0108] In cases where the scene moves (or the camera moves) (e.g., slower than a predetermined threshold), a temporal filter can be applied, for example, at the (single) bit level or at the full distance determination (i.e., at all bits).

[0109] In addition, measurement data from neighboring pixels (or multiple neighboring pixels) can be used for digital filters, or for bit-level or full-range determination, or both.

[0110] It should be noted that since bit-level operations can be performed according to this disclosure, bit-level filtering is also possible.

[0111] When stacking image sensors, filtering can be provided directly at the pixel level or in parallel layers.

[0112] By applying filters as described in this article, a reduction in lighting power can be achieved.

[0113] Furthermore, bit-level operations are often not as complex as operations on complex analog signals.

[0114] In some embodiments, macropixels are provided, thereby allowing the area of ​​the scene to spread across a grid on the image sensor (e.g., a three-by-three pixel grid, see also). Figure 3 (Discussion), where for each sub-pixel, its own demodulation function can be applied.

[0115] Therefore, each microframe can produce a full range determination that results in nine-bit resolution.

[0116] Therefore, ToF measurements can be performed efficiently through direct digital distance determination, which also saves processing power.

[0117] As is well known, for example in the iToF field, signals generated in a CAPD can be read by modulating one or more transmission gates of one or more transmission transistors included in or coupled to a CAPD.

[0118] According to some embodiments of this disclosure, a demodulation mode group can be used to perform demodulation of the transmission gate.

[0119] For example, in the case of a transmission gate, a first demodulation mode, such as a first demodulation frequency, may be applied during a first time interval, and a second demodulation mode, such as a second demodulation frequency, may be applied during a second time interval. This disclosure is not limited to a group of these two demodulation modes.

[0120] On the other hand, demodulation mode groups can be applied (approximately) simultaneously to multiple transmission gates.

[0121] In some embodiments, the time-of-flight circuit is further configured to mix at least one demodulated signal comprising a demodulation mode group with an optical detection signal indicating at least one detection event to generate a mixed signal, wherein the mixed signal encodes a predetermined time point.

[0122] As is generally known, at least one demodulation signal can be an electrical signal, such as a voltage signal, used to drive at least one transmission gate.

[0123] Optical detection signals can be voltage signals, current signals, power signals, digital signals, analog signals, etc., generated in response to optical detection events. For example, the photocurrent generated in response to light incident on the CAPD can be considered as an optical detection signal.

[0124] The optical detection signal and at least one demodulated signal can be electrically mixed, for example, by addition, multiplication, multiplexing, etc., to generate a result signal (i.e., a mixed signal).

[0125] As described above, a predetermined time point can be encoded by mixing signals.

[0126] For example, if the optical detection signal is within the logic high level of the demodulated signal, the mixed signal can have a value higher than a predetermined threshold so that it can encode 1, while if the optical detection signal is within the logic low level of the demodulated signal, the mixed signal can have a value lower than (or equal to) the predetermined threshold so that it can encode zero, and the present disclosure is not limited in this respect.

[0127] The temporal resolution of ToF detection can be affected (e.g., increased) by mixing the optical detection signal with multiple demodulated signals or by applying detection time interval groups separately.

[0128] In some embodiments, the time-of-flight circuitry is further configured to sequentially apply demodulation mode groups to at least one optical detection event.

[0129] Assuming that multiple detection events can be detected sequentially, and that the shapes of each optical detection signal are approximately the same, signal-to-noise ratio (SNR) coding can be performed effectively because for each measurement, the generated detection current can be detected only once, allowing the SNR to remain above a predetermined threshold.

[0130] However, in some embodiments, the time-of-flight circuit is also configured to apply the demodulation mode group to at least one optical detection event simultaneously.

[0131] Therefore, the optical detection signal (e.g., photocurrent) can be segmented to be detected at multiple transmission gates, i.e., simultaneously mixed with multiple demodulated signals, thereby enabling time-efficient encoding.

[0132] In some embodiments, the time-of-flight circuit is further configured to control the light source to emit light in a predetermined emission mode based on a predetermined detection mode.

[0133] The light source can be any light source suitable for the emission mode, such as diode lasers, LEDs (lasers), any kind of modulated light source, etc.

[0134] The predetermined emission mode can be based on the demodulation mode, and vice versa. The emission mode and demodulation mode can be correlated with each other in terms of their frequencies, i.e., for each emission pulse, there can be a corresponding demodulation pulse (i.e., detection time interval), but this disclosure is not limited thereto. The emission mode and demodulation mode can be further correlated in terms of their phases, or can be phase-shifted, so that the emission pulse can be detected within a demodulation pulse that takes into account the assumed flight time of the emitted light (which may depend on the detection distance, etc.).

[0135] However, frequency and / or phase correspondence may not be limited to a one-to-one correspondence, and it may depend on external factors, such as the maximum transmit frequency and the maximum demodulation frequency, pre-exposures that cannot be demodulated, etc.

[0136] Furthermore, the emission mode can have different shapes, such as short light pulses, square pulses, rectangular pulses, sawtooth pulses, etc., synchronized with the system period T (as described above).

[0137] In some embodiments, the optical pulse width is 25% of the system period T, but this disclosure is not limited to that particular value and may be implemented with any other optical pulse width.

[0138] For each demodulation function, a separate emission mode can be provided, resulting in a (maximum) n emission modes (which will also be discussed below). Figure 2 (Further discussion).

[0139] On the other hand, in embodiments that use short light pulses (i.e., light peaks) for each microframe, for example when the amplitude of the reflected light signal obtained from the direct optical path is greater than the amplitude (total) of the reflected light signal from the indirect optical path, degradation due to multipath operation can be reduced.

[0140] In addition, it can reduce nonlinearity and noise, while reducing average optical power (e.g., by 2 to 8 times), thereby saving power on the illumination side.

[0141] Some embodiments relate to photogate systems in which the detection time interval group can be applied by applying a voltage to a transparent gate (photogate) to direct photogenerated carriers to a corresponding detector adjacent to each transparent gate.

[0142] Some embodiments involve using the demodulation mode group to modulate the transmission gate of a transmission transistor, thereby applying the detection time interval group. Some embodiments relate to a time-of-flight method comprising: applying a detection time interval group to at least one optical detection event to determine a time point of at least one optical detection event, wherein the detection time interval group has a predetermined detection mode encoding a predetermined time point, as described herein.

[0143] As described in this article, this method can be executed via a time-of-flight circuit.

[0144] In some embodiments, as described herein, the predetermined detection mode is based on Gray code. In some embodiments, the time-of-flight method further includes applying the detection time interval group to a photon counter in the predetermined detection mode, the photon counter being coupled to a single-photon avalanche diode, as described herein. In some embodiments, as described herein, the photon counter is a Gray code counter. In some embodiments, as discussed herein, the time-of-flight method further includes decoding time points based on Gray code information generated in the Gray code counter indicating distance to the scene. In some embodiments, the time-of-flight method further includes using a demodulation current-assisted photon demodulator with a demodulation mode group to apply the detection time interval group, as discussed herein. In some embodiments, the time-of-flight method further includes mixing at least one demodulated signal including the demodulation mode group with a light detection signal indicating at least one detection event to generate a mixed signal, wherein the mixed signal encodes a predetermined time point, as described herein. In some embodiments, as discussed herein, the time-of-flight method further includes sequentially applying the demodulation mode group to at least one light detection event. In some embodiments, as described herein, the time-of-flight method further includes simultaneously applying the demodulation mode group to at least one light detection event. In some embodiments, the time-of-flight method further includes controlling a light source to emit light in a predetermined emission mode based on the predetermined detection mode.

[0145] In some embodiments, the methods described herein are also implemented as a computer program that, when executed on a computer and / or processor (which may be part of a ToF camera or system), causes the computer and / or processor to perform the methods described herein. In some embodiments, a non-transient computer-readable recording medium is also provided, wherein a computer program product is stored that, when executed by a processor such as the processor described above, causes the methods described herein to be performed.

[0146] Back Figure 1 In the case of an implementation in a CAPD-based image sensor, a timing sequence according to this disclosure is described. Figure 1 However, the timing itself is not limited to the CAPD case, as this timing can also be applied to SPAD-related embodiments. However, in this example, the timing... Figure 1 Including a demodulation mode for at least one transmission gate used to modulate the CAPD.

[0147] Time series Figure 1 It includes two detection cycles T. Each detection cycle begins with a light emission peak P. In response to the detection of the reflection of the light emission peak, a recording current I is generated. r (i.e., photocurrent). Peak light emission P and recording current I. r The time interval between the generation of light corresponds to the time of flight (ToF) of the emitted light.

[0148] The recording current I is detected using four demodulated signals f1, f2, f3, and f4. r Each demodulated signal includes its own detection time intervals T1, T2, T3 and T4, wherein the phase, frequency and number of detection time intervals of each demodulated signal f1 to f4 are different, and each demodulated signal is located between logic low (negative 1) and logic high (positive 1).

[0149] In this embodiment, f1 and f2 have the same number of detection time intervals (i.e., two), but different phases (90-degree phase shift), while the frequency of f3 is twice that of f1 and f2 (thus twice the number of detection time intervals) and is 90 degrees phase shifted relative to f2, and the frequency of f4 is twice that of f3 (thus twice the number of detection time intervals) and is 90 degrees phase shifted relative to f3.

[0150] Therefore, the demodulated signals f1 to f4 are Gray encoded.

[0151] By recording current I r Multiplying each demodulated signal f1 to f4 generates coded currents I1, I2, I3, and I4 (i.e., mixed signals as discussed herein), similar to the recording current I. rWhether it is located within the logic high level or logic low level of its respective demodulation function f1 to f4.

[0152] The encoding current I1 has a negative peak (i.e., a drop), similar to the recording current I. r Located within the logic low level of the demodulation signal f1, while the encoding currents I2 to I4 each have positive peak values, similar to the recording current I... r It is located within the logic high level of demodulation signals f2 to f4.

[0153] In this embodiment, the dropout is assigned to logic zero and the (positive) peak is assigned to logic 1, thereby encoding the recorded current Ir as a 0111 sequence, which indicates the minimum detection time interval in which the flight time of the emitted light peak is within this time interval.

[0154] Figure 2 The timing diagram for driving a light source according to this disclosure is shown. Figure 10 .

[0155] The light source is configured to output modulated light and is detected by an associated demodulated signal.

[0156] In this embodiment, the light source emits a first modulated light signal P. out1 This signal is detected by the first demodulated signal f1. In addition, the second modulated optical signal P is emitted. out2 The signal is detected by the second demodulated signal f2. The third modulated optical signal P is detected by the third demodulated signal f3. out3 The fourth demodulated optical signal P is detected by the fourth demodulated signal f4. out4 .

[0157] In this embodiment, the transmission and detection of each modulated optical signal are performed sequentially, that is, during the transmission and detection P... out2 Then execute P out2 The launch and detection (and in P) out2 P after that out3 Then execute P out4 This disclosure is not limited to this.

[0158] In this embodiment, P out2 With P out1 The same, therefore P out2 Similar to f2, but compared to f1, P out1 It is phase-shifted, which provides smooth operation.

[0159] Furthermore, in this embodiment, it is not necessary to generate short, intense light pulses for each period T, but it is sufficient to generate square (or rectangular) pulse waveforms with the same average power.

[0160] Figure 3A block diagram of the configuration of a macro image sensor 20 is shown, which includes a plurality of pixels 21, each pixel 21 including a CAPD.

[0161] Figure 3 The illustrated embodiment provides an implementation of simultaneously applying different demodulated signals f1 to f9.

[0162] For example, for each pixel 21, assume that the same (roughly the same) recording current is generated (as referenced). Figure 1 (i.e., the signal shape and intensity of the recorded current can be approximately the same).

[0163] Therefore, for each pixel, different demodulated signals f1 to f9 can be applied to simultaneously generate the encoded current (as referenced). Figure 1 (As discussed), in this way, the ToF measurement performed using the image sensor according to this embodiment can be performed with time efficiency and signal-to-noise ratio efficiency.

[0164] Figure 4 A block diagram 30 depicts the readout circuitry for the pixel 31, which includes a SPAD.

[0165] The current signal generated in pixel 31 is sampled (32), i.e., the well-known time-to-digital conversion, wherein the conversion is timed with the time-to-digital converter (TDC) clock 33.

[0166] In response to sample 32, a signal is sent to the Gray Code (GC) counter 34 controlled by the Gray Code GC controller 35, in a manner consistent with... Figure 1 The timing sequence of the demodulation signals f1 to f4 described is applied accordingly. Therefore, if the photocurrent is detected within the detection time, if the photocurrent is detected, logic 1 is stored in GC bit memory 36; if the photocurrent is not detected, logic 0 is stored in GC bit memory 36.

[0167] Reference Figure 4 In the described embodiments, the timing of subsequent measurements can be different (e.g., the first timing may correspond to the timing of the demodulated signal f1, and the second timing may correspond to...). Figure 1 (such as the timing of the demodulated signal f2, etc.) so that sequential processing is performed in this embodiment.

[0168] On the other hand, Figure 5 Block diagram 40 illustrates an embodiment of performing parallel processing.

[0169] Figure 5 Implementation examples and Figure 4The difference in the embodiments is that there are multiple (at least two) GC counters controlled by multiple GC controllers, and these GC counters are controlled simultaneously, i.e. in parallel, so that ToF measurements can be performed efficiently.

[0170] Figure 6 A block diagram 50 is depicted for driving the Gray code counter 51. As described above, a sampling signal 52 from the sampler and a control signal 53 from the Gray code controller are fed to the Gray code counter.

[0171] According to the timing sequence described above, control signal 53 sets control switch 54 to one of two positions. In the first position, switch 54 is coupled to Gray code 0 counter 55 (GC 0). In the second position, switch 54 is coupled to Gray code 1 counter 56 (GC1).

[0172] The result signals from Gray code 0 counter 55 and Gray code 1 counter 56 are compared in comparator 57, and the result GC value is generated in GC value generation unit 58. If a GC bit code has already been generated, the GC value is appended to the existing GC bit code in GC bit value generation unit 59, or if a GC bit code has not yet been generated, the GC value is initialized as a GC bit code.

[0173] Figure 7 Block diagram 60 depicts another embodiment of feeding a sampling signal 62 and a control signal 63 to a GC counter 61, wherein the sampling signal 62 and the control signal 63 are compared by a comparator 64, and the resulting value is fed to a GC value generation unit 65, as described above. Figure 6 The GC value generation unit 65 feeds its results to the GC bit value generation unit 66.

[0174] Figure 8 Block diagram 70 depicts another embodiment of feeding a sampling signal 72 and a control signal 73 to a GC counter 71. The sampling signal 72 is fed to a bidirectional shift register 74, and the control signal 73 is fed to a forward / reverse unit 75, which in turn feeds its signal to the bidirectional shift register 74 to control the direction of bit shifting in the bidirectional shift register, which is configured to compare the two signals and generate Gray code values.

[0175] Then, the generated GC value is provided to the GC bit value generation unit 76.

[0176] Figure 9 Figure 77 shows a data compression diagram according to this disclosure.

[0177] Histogram 78 is shown for illustrative purposes only, as is well known in the dToF field. However, it should be noted that generating a histogram is not required according to this disclosure. As mentioned at the beginning, such a histogram may consume a certain amount of memory. Histograms typically include the measured round-trip delay (i.e., time) on the x-axis and the number of times that round-trip delay was detected (i.e., count) on the y-axis, as is well known.

[0178] To reduce the amount of memory required, the histogram is Gray-coded according to this disclosure, as shown in Gray-coded histogram 79, which includes the same axes as histogram 78. Since Gray coding is discussed below, neither histogram 78 nor Gray-coded histogram 79 needs to be generated. Histograms 78 and Gray-coded histogram 79 are for illustrative purposes only.

[0179] From the five-bit Gray code histogram in this embodiment, the Gray code can be read, which indicates the round-trip delay and thus the distance to the scene.

[0180] Figure 10 A block diagram of method 80 according to this disclosure is depicted.

[0181] In 81, as discussed herein, a detection time interval group is applied to the photon counter.

[0182] In 82, as discussed herein, the predetermined time point indicating the distance to the scene is decoded based on the detection of the application based on the detection time interval group.

[0183] Figure 11 A block diagram of method 90 according to this disclosure is depicted.

[0184] In 91, as discussed herein, a demodulation mode group including a detection time interval is applied to the transmission gate.

[0185] In 92, as discussed herein, a mixed signal (or multiple mixed signals) is generated by mixing the current generated in response to a detection event with a demodulation mode.

[0186] Figure 12 A block diagram of method 90', which is a first alternative to method 90, is depicted. The difference between method 90 and method 90 is that 91' replaces 91, wherein, as discussed herein, demodulation mode groups are applied sequentially.

[0187] Figure 13 A block diagram of "Method 90", an alternative to Method 90, is depicted, which differs from Method 90 in that, as discussed herein, 91 is replaced with 91", in which a demodulation mode group is applied simultaneously.

[0188] Figure 14A block diagram of method 100, including method 90, is depicted. Furthermore, as described herein, method 100 includes, at 101, controlling a light source to emit light in a predetermined emission mode.

[0189] Figure 15 A block diagram of a ToF camera 110 according to this disclosure is depicted.

[0190] The ToF camera 110 includes an image sensor 111 based on multiple SPADs and an encoding unit 112 configured to sample the image sensor and encode light detection events, as described above. Figures 4 to 8 Any of the above discussed herein. Furthermore, as described herein, the ToF camera 110 includes a ToF circuit 113 configured to apply a detection time interval to the encoding unit 112. The ToF camera 110 also includes a light source configured to emit light reflected from the scene and detected by the image sensor 111.

[0191] Figure 16 A block diagram of a ToF camera 120 according to this disclosure is depicted.

[0192] The ToF camera 120 includes an image sensor 121 based on multiple CAPDs and a demodulation unit 122 configured to sample the transmission gates (or multiple transmission gates) of the image sensor by applying one or more demodulated signals and to encode light detection events, as described above. Figures 1 to 3 Any of the above discussed. Furthermore, the ToF camera 120 includes a ToF circuit 123 configured to apply a detection time interval to the demodulation unit 122 and control the demodulation unit 122 to sample the image sensor 121, as described herein. The ToF camera 120 also includes a light source configured to emit light reflected from the scene and detected by the image sensor 121.

[0193] As described herein, the ToF circuit 123 is also configured to control the light source to emit light in a light emission mode.

[0194] It should be understood that the embodiments describe an exemplary sequence of method steps. However, the specific order of the method steps is for illustrative purposes only and should not be construed as binding. For example, the steps can be interchanged. Figure 13 The order of 101 and 91 in the embodiments is shown. Other variations in the order of method steps may be obvious to those skilled in the art.

[0195] Please note that the division of the ToF camera 110 or 120 into units 111 to 114 or 121 to 124 is for illustrative purposes only, and this disclosure is not limited to any particular functional division within a particular unit. For example, the demodulation unit 122 and the ToF circuit 123 may be implemented by their respective programming processors, field-programmable gate arrays (FPGAs), etc.

[0196] Unless otherwise stated, all units and entities described in this specification and claimed in the appended claims can be implemented as integrated circuit logic, for example on a chip, and unless otherwise stated, the functionality provided by these units and entities can be implemented by software.

[0197] When the above-disclosed embodiments are implemented at least in part using a software-controlled data processing device, it should be understood that the computer program providing such software control and transmission, and the storage or other medium providing such computer program, are contemplated as aspects of this disclosure.

[0198] Note that this technology can also be configured as described below.

[0199] (1) The time-of-flight circuit is configured as follows:

[0200] A detection time interval group is applied to at least one optical detection event to determine the time point of at least one optical detection event, wherein the detection time interval group has a predetermined detection pattern that encodes the predetermined time point.

[0201] (2) The time-of-flight circuit according to (1), wherein the predetermined detection mode is based on Gray code.

[0202] (3) The time-of-flight circuit according to any one of (1) and (2) is further configured to apply the detection time interval group to a photon counter in a predetermined detection mode, the photon counter being coupled to a single photon avalanche diode.

[0203] (4) According to the time-of-flight circuit described in (3), the photon counter is a Gray code counter.

[0204] (5) The time-of-flight circuit according to any one of (3) and (4) is further configured to decode time points based on Gray code information indicating the distance to the scene generated in the Gray code counter.

[0205] (6) The time-of-flight circuit according to any one of (1) to (3) is further configured to sequentially apply the detection time interval group to at least one optical detection event.

[0206] (7) The time-of-flight circuit according to any one of (1) to (3) is further configured to simultaneously apply the detection time interval group to at least one optical detection event.

[0207] (8) The time-of-flight circuit according to any one of (1) and (2) is further configured as follows:

[0208] The demodulation mode group uses a demodulation current-assisted photon demodulator to apply the detection time interval group.

[0209] (9) The time-of-flight circuit according to (8) is further configured to mix at least one demodulated signal including the demodulation mode group with an optical detection signal indicating the at least one detection event to generate a mixed signal, wherein the mixed signal encodes a predetermined time point.

[0210] (10) The time-of-flight circuit according to any one of (8) and (9) is further configured to sequentially apply the demodulation mode group to at least one optical detection event.

[0211] (11) The time-of-flight circuit according to any one of (8) and (9) is further configured to simultaneously apply the demodulation mode group to at least one optical detection event.

[0212] (12) The time-of-flight circuit according to any one of (8) to (11) is further configured to control the light source to emit light in a predetermined emission mode based on a predetermined detection mode.

[0213] (13) A time-of-flight method, comprising:

[0214] A detection time interval group is applied to at least one optical detection event to determine the time point of at least one optical detection event, wherein the detection time interval group has a predetermined detection pattern that encodes the predetermined time point.

[0215] (14) According to the time-of-flight method described in (13), wherein the predetermined detection mode is based on Gray code.

[0216] (15) The time-of-flight method according to any one of (13) and (14) further includes applying the detection time interval group to a photon counter in a predetermined detection mode, the photon counter being coupled to a single photon avalanche diode.

[0217] (16) According to the time-of-flight method described in (15), wherein the photon counter is a Gray code counter.

[0218] (17) The time-of-flight method according to any one of (15) and (16) further includes decoding the time point based on Gray code information indicating the distance to the scene generated in the Gray code counter.

[0219] (18) The time-of-flight method according to any one of (13) to (15) further includes: sequentially applying the detection time interval group to the at least one optical detection event.

[0220] (19) The time-of-flight method according to any one of (13) to (15) further includes: simultaneously applying the detection time interval group to the at least one optical detection event.

[0221] (20) According to the time-of-flight method of (13), it also includes using a demodulation mode group to demodulate the photon demodulator with a demodulation current to apply the detection time interval group.

[0222] (21) The time-of-flight method according to (20) further includes mixing at least one demodulated signal including the demodulation mode group with an optical detection signal indicating the at least one detection event to generate a mixed signal, wherein the mixed signal encodes the predetermined time point.

[0223] (22) The time-of-flight method according to any one of (20) and (21) further includes sequentially applying the demodulation mode group to the at least one optical detection event.

[0224] (23) The time-of-flight method according to (20) and (21) further includes simultaneously applying the demodulation mode group to the at least one optical detection event.

[0225] (24) The time-of-flight method according to any one of (20) to (23) further includes controlling the light source to emit light in a predetermined emission mode based on the predetermined detection mode.

[0226] (25) A computer program comprising, when executed on a computer, program code that causes the computer to perform the method according to any one of (13) to (24).

[0227] (26) A non-transient computer-readable recording medium storing a computer program product which, when executed by a processor, causes to perform the method according to any one of (13) to (24).

Claims

1. A time-of-flight circuit configured to: applying a set of detection time intervals to at least one light detection event to determine a time point of the at least one light detection event, wherein the set of detection time intervals has a predetermined detection pattern encoding a predetermined time point, wherein apply a set of detection time intervals to at least one light detection event to determine a time point of the at least one light detection event, wherein the set of detection time intervals has a predetermined detection pattern encoding a predetermined time point, wherein the predetermined detection pattern is based on a Gray code, apply the set of detection time intervals of the predetermined detection pattern to a Gray code counter coupled to a single photon avalanche diode, and decode the time point based on Gray code information generated in the Gray code counter indicating a distance to a scene.

2. The time-of-flight circuit of claim 1, further configured to apply the set of detection time intervals sequentially to the at least one light detection event.

3. The time-of-flight circuit of claim 1, further configured to apply the set of detection time intervals simultaneously to the at least one light detection event.

4. The time-of-flight circuit of claim 1, further configured to: demodulate a current assisted photonic demodulator using a set of demodulation patterns to thereby apply the set of detection time intervals.

5. The time-of-flight circuit of claim 4, further configured to mix at least one demodulation signal comprising the set of demodulation patterns with a light detection signal indicative of the at least one detection event to thereby generate a mixed signal, wherein the mixed signal encodes the predetermined time point.

6. The time-of-flight circuit of claim 4, further configured to apply the set of demodulation patterns sequentially to the at least one light detection event.

7. The time-of-flight circuit of claim 4, further configured to apply the set of demodulation patterns simultaneously to the at least one light detection event.

8. The time-of-flight circuit of claim 4, further configured to control a light source to emit light in a predetermined emission pattern based on the predetermined detection pattern.

9. A time-of-flight method comprising: applying a set of detection time intervals to at least one light detection event to determine a time point of the at least one light detection event, wherein the set of detection time intervals has a predetermined detection pattern encoding a predetermined time point, wherein the predetermined detection pattern is based on a Gray code, applying the set of detection time intervals of the predetermined detection pattern to a Gray code counter coupled to a single photon avalanche diode, and decoding the time point based on Gray code information generated in the Gray code counter indicating a distance to a scene.

10. The time-of-flight method of claim 9, further comprising: applying the set of detection time intervals sequentially to the at least one light detection event.

11. The time-of-flight method of claim 9, further comprising: applying the set of detection time intervals simultaneously to the at least one light detection event.

12. The time-of-flight method of claim 9, further comprising demodulating a current assisted photonic demodulator using a set of demodulation patterns to thereby apply the set of detection time intervals.

13. The time-of-flight method of claim 12, further comprising mixing at least one demodulation signal comprising the set of demodulation patterns with a light detection signal indicative of the at least one detection event to thereby generate a mixed signal, wherein the mixed signal encodes the predetermined time point.

14. The time-of-flight method of claim 12, further comprising applying the set of demodulation patterns sequentially to the at least one light detection event.

15. The time-of-flight method of claim 12, further comprising simultaneously applying the set of demodulation patterns to the at least one light detection event.

16. The time-of-flight method of claim 12, further comprising controlling a light source to emit light in a predetermined emission pattern based on the predetermined detection pattern.

Images