Time-of-flight sensor and method for measuring distance using the same

Abstract

A method measures the distance between a time-of-flight (ToF) sensor and an object based on a cross-time offset. The ToF sensor includes at least one depth pixel and a light source for directing emitted light to the object. The depth pixel can have a multi-tap structure and can generate sampling data based on received light and a demodulated signal with different phases. The received light corresponds to emitted light reflected from the object. The method includes: generating a time offset between the emitted light and the demodulated signal; performing a sampling operation to generate sampling data corresponding to the time offset; determining the cross-time offset based on the sampling data of the first reference tap being substantially equal to the sampling data of the second reference tap; and determining the distance between the ToF sensor and the object based on the cross-time offset.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from Korean Patent Application No. 10-2020-0098448 filed on August 6, 2020, with the Korean Intellectual Property Office (KIPO), the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Example embodiments relate to a time-of-flight sensor and a method of measuring distance using the time-of-flight sensor. Background Art

[0004] Various sensing technologies have been developed to obtain three-dimensional information about an object. Some of these technologies involve the use of three-dimensional cameras, including time-of-flight (ToF) sensors. Such sensors can illuminate an object with emitted light and then calculate the distance to the object by measuring the time of flight or phase difference of the light reflected from the object using the demodulated signal. The accuracy of the resulting distance measurement can be degraded due to various effects, including but not limited to offset or gain deviation of the depth pixels in the ToF sensor. Summary of the Invention

[0005] Some example embodiments may provide a time-of-flight (ToF) sensor capable of improving the accuracy of distance measurement.Some example embodiments may also provide a method of measuring distance using the ToF sensor.

[0006] According to one or more embodiments, a method for measuring distance using a time-of-flight (ToF) sensor includes at least one depth pixel and a light source that irradiates an object with emitted light. The depth pixel has a multi-tap structure including multiple taps to generate multiple sampling data based on received light and multiple demodulated signals with different phases. The emitted light is reflected from the object and returned to the ToF sensor as received light.

[0007] The method includes: generating a plurality of time offsets between emitted light and a plurality of demodulated signals; performing a plurality of sampling operations to generate a plurality of sampling data corresponding to the plurality of time offsets; determining a cross time offset based on the plurality of sampling data corresponding to the plurality of time offsets so that sampling data of a first reference tap and a second reference tap among a plurality of taps become substantially equal to each other with respect to the cross time offset; and determining a distance between a ToF sensor and an object based on the cross time offset.

[0008] According to one or more embodiments, a method for measuring distance using a time-of-flight (ToF) sensor includes at least one depth pixel and a light source that irradiates an object with emitted light. The depth pixel has a multi-tap structure including multiple taps to generate multiple sampling data based on received light and multiple demodulated signals with different phases. The emitted light is reflected from the object and returned to the ToF sensor as received light.

[0009] The method includes: generating a first time offset between emitted light and a plurality of demodulated signals; performing a first sampling operation to generate first sampling data corresponding to the first time offset; generating a second time offset between the emitted light and the plurality of demodulated signals; performing a second sampling operation to generate second sampling data corresponding to the second time offset; determining a cross time offset based on the first sampling data and the second sampling data so that sampling data of a first reference tap and a second reference tap among a plurality of taps become substantially equal to each other with respect to the cross time offset; and determining a distance between a ToF sensor and an object based on the cross time offset.

[0010] According to one or more embodiments, a time-of-flight (ToF) sensor includes: a light source configured to irradiate an object with emitted light; and a sensor including a pixel array and a row scanning circuit. The pixel array includes one or more depth pixels, each of the one or more depth pixels having a multi-tap structure including multiple taps, and is configured to generate multiple sampling data based on received light and multiple demodulation signals with different phases. The emitted light is reflected from the object back to the ToF sensor as received light, and the row scanning circuit is configured to generate multiple demodulation signals applied to the multiple taps.

[0011] The ToF sensor further includes a controller configured to perform a plurality of sampling operations by applying a plurality of demodulation signals having a plurality of time offsets relative to the emitted light, thereby generating a plurality of sampled data corresponding to the plurality of time offsets. Furthermore, the controller is configured to determine a cross-time offset based on the plurality of sampled data corresponding to the plurality of time offsets, such that the sampled data of a first reference tap and a second reference tap among the plurality of taps become substantially equal to each other with respect to the cross-time offset, and to determine a distance between the ToF sensor and the object based on the cross-time offset. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Example embodiments of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0013] Figure 1 An embodiment of a method of measuring distance using a ToF sensor is shown.

[0014] Figure 2 An embodiment of a ToF sensor is shown.

[0015] Figure 3 and Figure 4 Shown are example methods for measuring and calculating distance to objects.

[0016] Figure 5 and Figure 6 An example embodiment showing time offset of a ToF sensor.

[0017] Figure 7 An example embodiment of a method of determining a cross-time offset for measuring distance is shown.

[0018] Figures 8A to 8F An embodiment of sampling data according to time shift of a ToF sensor is shown.

[0019] Figure 9 An embodiment of a depth pixel with a two-tap structure in a ToF sensor is shown.

[0020] Figure 10 An example operation of a ToF sensor including depth pixels is shown.

[0021] Figure 11 An embodiment of a depth pixel with a two-tap structure in a ToF sensor is shown.

[0022] Figure 12 An example operation of a ToF sensor including depth pixels is shown.

[0023] Figure 13 An embodiment of a pulse-based approach to ToF is shown.

[0024] Figure 14 An embodiment of a continuous wave based scheme for ToF is shown.

[0025] Figure 15 and Figure 16 An embodiment of a time-shifted scanning method for a ToF sensor is shown.

[0026] Figure 17 and Figure 18 An example embodiment of a sampling operation of a ToF sensor is shown.

[0027] Figure 19 and Figure 20 An example embodiment of time offset for distance measurement by a ToF sensor is shown.

[0028] Figure 21 An embodiment of a method of measuring distance is shown.

[0029] Figure 22 and Figure 23 is used to describe Figure 21 An example diagram of the method.

[0030] Figure 24 A circuit embodiment of a depth pixel with a four-tap structure in a ToF sensor is shown.

[0031] Figure 25 An example operation of a ToF sensor including depth pixels is shown.

[0032] Figure 26 A circuit embodiment of a depth pixel with a four-tap structure in a ToF sensor is shown.

[0033] Figure 27 An example operation of a ToF sensor including depth pixels is shown.

[0034] Figure 28 and Figure 29 An embodiment showing time offset of a ToF sensor is shown.

[0035] Figure 30 An example of sampling data according to time shift of a ToF sensor is shown.

[0036] Figure 31 An embodiment of a time-shifted scanning method for a ToF sensor is shown.

[0037] Figure 32 An embodiment of a computing system including a ToF sensor is shown. DETAILED DESCRIPTION

[0038] Various example embodiments will be described more fully below with reference to the accompanying drawings, in which some example embodiments are shown. In the accompanying drawings, like reference numerals denote like elements throughout. Repetitive descriptions may be omitted. According to one or more embodiments, the term "substantially" may correspond to a predetermined tolerance.

[0039] Figure 1 is a flowchart illustrating a method for measuring distance using time of flight (ToF) according to an example embodiment. According to an example embodiment, the ToF sensor may include at least one depth pixel and may further include or be coupled to a light source for providing emitted light for illuminating an object. In one embodiment, the depth pixel may have a multi-tap structure (including multiple taps) to generate multiple sampling data based on received light and demodulated signals with different phases. The received light may correspond to the emitted light that has been reflected from the object back to the ToF sensor. The method for measuring distance according to an example embodiment may be performed using such a ToF sensor.

[0040] refer to Figure 1The method includes: at S100, generating a plurality of time offsets between the emitted light and the plurality of demodulated signals. The time offsets may indicate a relative time difference or phase difference between the emitted light and the demodulated signals. Figure 4 and Figure 5 In some example embodiments described in (e.g., reference to) the time offset can be changed by adjusting the delay amount or phase of the emitted light and / or the demodulated signal. Figure 19 and Figure 20 In some example embodiments described herein, the time offset may be changed by adjusting the pulse width or duty cycle of the demodulated signal.

[0041] At S200 , multiple sampling operations may be performed to generate multiple sampling data corresponding to the time offset. As described below, the sampling operations may be performed using various methods depending on, for example, the tap structure of the depth pixel, the modulation scheme of the emitted light, and / or other characteristics.

[0042] At S300 , a cross time offset may be determined based on a plurality of sampling data corresponding to a plurality of time offsets so that sampling data of a first reference tap and a second reference tap among a plurality of taps become equal to each other with respect to the cross time offset.

[0043] At S400, the distance between the ToF sensor and the object may be determined based on the cross time offset. Figures 7 to 8F Example embodiments for determining cross-time offsets and distance calculations are described.

[0044] According to one approach, a ToF sensor can determine the distance between the ToF sensor and an object based on the ratio of sampled data corresponding to a demodulated signal. Such an approach can produce varying or inaccurate distance calculations depending on the offset or gain deviation of the depth pixels in the ToF sensor.

[0045] According to example embodiments, a cross-time offset corresponding to an intersection of sampled data can be determined based on a change in time offset. The cross-time offset can then be used as a basis for determining the distance between the ToF sensor and the object. Using this method and other methods described herein, an accurate distance can be obtained even when the characteristic deviation of one or more depth pixels varies. That is, by using a cross-time offset determined based on multiple sampled data corresponding to different time offsets, the ToF sensor and method for measuring distance according to example embodiments can improve the accuracy of distance measurement regardless of the characteristic deviation of the depth pixels.

[0046] In (for example, reference Figure 22 and Figure 23In some example embodiments described herein, the approach velocity of an object may be determined based on a plurality of sampled data corresponding to a plurality of time offsets. In this case, the sampled data may be corrected based on the approach velocity, and the cross-time offset may be corrected based on the corrected sampled data. Therefore, the ToF sensor and method for measuring distance according to example embodiments may measure the approach velocity of an object based on the sampled data, and the measured approach velocity may be used to further improve the accuracy of distance measurement.

[0047] Figure 2 is a block diagram illustrating an embodiment of a ToF sensor 100 , which may include a sensing unit (sensor), a controller 150 , and a light source module 200 .

[0048] refer to Figure 2 The sensing unit (sensor) may include a pixel array 110, an analog-to-digital converter (ADC) unit 120, a row scanning circuit 130, and a column scanning circuit 140. The pixel array 110 may include one or more depth pixels that receive light RL reflected from the object OBJ. The light reflected from the object OBJ is based on the light emitted toward the object OBJ by the light source module 200. The depth pixel may convert the received light RL into an electrical signal. The depth pixel may then provide information related to the distance of the object OBJ from the ToF sensor 100.

[0049] In one embodiment, the pixel array 110 may include color pixels for providing color image information. In this case, the ToF sensor 100 may be a three-dimensional color image sensor that provides color image information and depth information. According to example embodiments, an infrared filter and / or a near-infrared filter may be formed on the depth pixels, and a color filter (e.g., a red filter, a green filter, and a blue filter) may be formed on the color pixels. According to example embodiments, the ratio of the number of depth pixels to the number of color pixels may vary as needed or by design.

[0050] The ADC unit 120 may convert an analog signal output from the pixel array 110 into a digital signal. According to an example embodiment, the ADC unit 120 may perform column analog-to-digital conversion, which converts analog signals in parallel using a plurality of analog-to-digital converters respectively coupled to a plurality of column lines. According to an example embodiment, the ADC unit 120 may perform single analog-to-digital conversion, which sequentially converts analog signals using a single analog-to-digital converter.

[0051] According to example embodiments, the ADC unit 120 may include a correlated double sampling (CDS) unit for extracting a valid signal component. In some example embodiments, the CDS unit may perform analog double sampling, which extracts the valid signal component based on the difference between an analog reset signal including a reset component and an analog data signal including a signal component. In some example embodiments, the CDS unit may perform digital double sampling, which converts the analog reset signal and the analog data signal into two digital signals and extracts the valid signal component based on the difference between the two digital signals. In some example embodiments, the CDS unit may perform dual correlated double sampling, which performs both analog double sampling and digital double sampling.

[0052] The row scanning circuit 130 may receive a control signal from the controller 150 and may control a row address and row scanning of the pixel array 110. In order to select a row line among a plurality of row lines, the row scanning circuit 130 may apply a signal for activating the selected row line to the pixel array 110. According to example embodiments, the row scanning circuit 130 may include a row decoder for selecting a row line of the pixel array 110 and a row driver for applying a signal for activating the selected row line.

[0053] The column scan circuit 140 may receive a control signal from the controller 150 and may control the column address and column scan of the pixel array 110. The column scan circuit 140 may output the digital output signal from the ADC unit 120 to a digital signal processing circuit and / or to an external host. For example, the column scan circuit 140 may provide a horizontal scan control signal to the ADC unit 120 to sequentially select a plurality of analog-to-digital converters in the ADC unit 120.

[0054] The controller 150 may control the ADC unit 120, the row scanning circuit 130, the column scanning circuit 140, and the light source module 200. The controller 150 may provide a control signal, such as at least one of a clock signal, a timing control signal, or another signal, to the ADC unit 120, the row scanning circuit 130, the column scanning circuit 140, and the light source module 200. The controller 150 may include at least one of a control logic circuit, a phase-locked loop circuit, a timing control circuit, a communication interface circuit, or another circuit.

[0055] The light source module 200 can emit light of a desired (or predetermined) wavelength. For example, the light source module 200 can emit infrared light and / or near-infrared light. The light source module 200 may include a light source 210 and a lens 220. The light source 210 may be controlled by the controller 150 to emit light TL of a desired intensity and / or characteristic (e.g., periodicity). For example, the intensity and / or characteristic of the emitted light TL may be controlled so that the emitted light TL has a predetermined waveform, such as a pulse wave, a sine wave, a cosine wave, or another type of wave. The light source 210 may be implemented by a light emitting diode (LED), a laser diode, or another type of light source.

[0056] A normal operation of the ToF sensor 100 according to example embodiments will now be described below.

[0057] The controller 150 can control the light source module 200 to emit emission light TL having a periodic intensity. The emission light TL emitted by the light source module 200 can be reflected from the object OBJ and returned to the ToF sensor 100 as received light RL. The received light RL can be incident on the depth pixels, and the depth pixels can be activated by the row scanning circuit 130 to output analog signals corresponding to the received light RL. The ADC unit 120 can convert the analog signals output from the depth pixels into sampled data SDATA. The sampled data SDATA can be provided to the controller 150 by the column scanning circuit 140 and / or the ADC 120.

[0058] The controller 150 (or an external processor) can calculate the distance of the object OBJ from the ToF sensor 100, the horizontal position of the object OBJ, the vertical position of the object OBJ, and / or the size of the object OBJ based on the sampled data SDATA. The controller 150 can control the emission angle or projection (or incident) area of ​​the emitted light TL based on the distance, horizontal position, vertical position, and / or size of the object OBJ. For example, the controller 150 can control the spacing between the light source 210 and the lens 220, the relative position (or arrangement) of the light source 210 and the lens 220 relative to each other, the refractive index of the lens 220, the curvature of the lens 220, or another feature of the lens 220.

[0059] The emission light TL irradiated to the object OBJ may be reflected, and the reflected light (e.g., received light RL) may be incident on the depth pixels in the pixel array 110. The depth pixels may output analog signals corresponding to the received light RL, and the ADC unit 120 may convert the analog signals into digital data or sampled data SDATA. The sampled data SDATA and / or the depth information may be provided to the controller 150, the digital signal processing circuit, and / or the external host. According to example embodiments, the pixel array 110 may include color pixels, and the color image information and the depth information may be provided to the digital signal processing circuit and / or the external host.

[0060] Figure 3 and Figure 4 is a diagram for describing an example method of measuring and calculating a distance to an object according to an embodiment.

[0061] refer to Figure 2 and Figure 3 , the emission light TL emitted by the light source module 200 may have (e.g., predetermined) periodic intensity and / or characteristics. For example, the intensity of the emission light TL (e.g., the number of photons per unit area) may have a predetermined waveform, which in this example is a sine wave, but may be a different type of waveform in another embodiment.

[0062] The emission light TL emitted by the light source module 200 may be reflected from the object OBJ and then may be incident on the pixel array 110 as the received light RL. The pixel array 110 may periodically sample the received light RL. According to an example embodiment, during each cycle of the received light RL (e.g., which may correspond to the cycle of the emitted light TL), the pixel array 110 may perform sampling on the received light RL by, for example, sampling at two sampling points with a predetermined phase difference (e.g., approximately 180 degrees), at four sampling points with a phase difference (e.g., approximately 90 degrees), or at more than four sampling points. For example, the pixel array 110 may extract four samples A0, A1, A2, and A3 of the received light RL at phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees of each cycle, respectively.

[0063] Due to background light, noise, and / or other influences, the received light RL may have an offset B different from an offset of the emission light TL emitted by the light source module 200. The offset B of the received light RL may be calculated based on Equation 1.

[0064]

[0065] Among them, A0 represents the intensity of the received light RL sampled at a phase of approximately 90 degrees of the transmitted light TL, A1 represents the intensity of the received light RL sampled at a phase of approximately 180 degrees of the transmitted light TL, A2 represents the intensity of the received light RL sampled at a phase of approximately 270 degrees of the transmitted light TL, and A3 represents the intensity of the received light RL sampled at a phase of approximately 360 degrees of the transmitted light TL.

[0066] Due to loss (eg, light loss), the received light RL may have a lower amplitude A than the amplitude of the emission light TL emitted by the light source module 200. The amplitude A of the received light RL may be calculated based on Formula 2.

[0067]

[0068] Black and white image information corresponding to the object OBJ may be provided by each depth pixel in the pixel array 110 based on the amplitude A of the received light RL.

[0069] The received light RL may be delayed relative to the transmitted light TL by a phase difference Φ corresponding to, for example, twice the distance of the object OBJ from the ToF sensor 100. The phase difference Φ between the transmitted light TL and the received light RL may be calculated based on Equation 3.

[0070]

[0071] The phase difference Φ between the transmitted light TL and the received light RL can correspond to, for example, the time of flight (ToF). In one embodiment, the distance of the object OBJ from the ToF sensor 100 can be calculated using the following formula: R = c*ToF / 2, where R represents the distance of the object OBJ and c represents the speed of light. Furthermore, the phase difference Φ between the transmitted light TL and the received light RL can be used to calculate the distance of the object OBJ from the ToF sensor 100 based on Formula 4.

[0072]

[0073] Here, f represents a modulation frequency, which may be, for example, the frequency of the intensity of the transmitted light TL or the frequency of the intensity of the received light RL.

[0074] exist Figure 3 , an example in which the intensity of the emitted light TL has a sinusoidal waveform is shown. In other embodiments, the intensity of the emitted light TL may have other types of waveforms. Furthermore, the ToF sensor 100 may extract depth information based on the waveform of the intensity of the emitted light TL, the structure of the depth pixel, and / or another feature or parameter.

[0075] Figure 4 This section illustrates exemplary embodiments of modulation and demodulation timing for depth pixels with a multi-tap structure. In a two-tap structure, the first and second demodulated signals DEM1 and DEM2 can be different from the third and fourth demodulated signals DEM3 and DEM4, for example, through time division. In a four-tap structure, the first through fourth demodulated signals DEM1 through DEM4 can be simultaneously applied to the same depth pixel. The duty cycle of the four-tap demodulated signals DEM1 through DEM4 can be reduced to less than half that of the two-tap structure. Figure 4 An example of the modulation timing corresponding to the phase of the emission light TL and the demodulation timing corresponding to the phase of the demodulation signals DEM1 to DEM4 is described. The operation of the ToF sensor may be modified in various ways in other embodiments.

[0076] refer to Figure 4 , the emission light TL from the light source 210 can be output synchronously with the signal from the controller 150. The first demodulation signal DEM1 to the fourth demodulation signal DEM4 can be generated synchronously with the signal from the controller 150. The first demodulation signal DEM1 to the fourth demodulation signal DEM4 have phase differences of 0 degrees, 90 degrees, 180 degrees and 270 degrees respectively. As mentioned above Figure 3 As described above, the four samples A0, A1, A2 and A3 of the received light RL can be sampled at phases of 0 degrees, 90 degrees, 180 degrees and 270 degrees in each cycle, respectively. In another embodiment, the sampling phases can be different.

[0077] exist Figure 4 , an example is shown in which the phase of the first demodulation signal DEM1 is consistent with the phase of the emission light TL. In one embodiment, the first to fourth demodulation signals DEM1 to DEM4 can be applied to the first to fourth demodulation transmission gates, respectively, as described below.

[0078] Although providing Figure 3 and Figure 4 An example of the principle of using a ToF sensor to measure and calculate the distance to an object is described, but it should be understood that one or more other example embodiments are not limited thereto. For example, in other embodiments, the duty cycle of the emitted light TL and / or the number, phase difference, and / or duty cycle of the demodulated signal may be the same as that of the same sensor. Figure 3 and Figure 4 The corresponding ones are different.

[0079] Figure 5 and Figure 6 is a diagram illustrating a time offset of a ToF sensor according to an example embodiment. Figure 5 and Figure 6 In FIG, the following are shown based on an example of a two-tap structure: the timing of the transmitted light TL, the received light RL corresponding to an arbitrary time of flight ToF, the first demodulation signal DEM1 corresponding to the first tap, and the second demodulation signal DEM2 corresponding to the second tap. Figure 5 An example in which the time offset Ts is 0 is shown in Figure 6 An example in which the time offset Ts has a positive value is shown in FIG.

[0080] When the depth pixel has a two-tap structure including a first tap and a second tap, the first tap may correspond to a first reference tap and the second tap may correspond to a second reference tap, regardless of time of flight ToF (eg, a distance between a ToF sensor and an object).

[0081] refer to Figure 5 and Figure 6, the pulse width of the transmitted light TL and the pulse widths of the plurality of demodulated signals may be equal to each other as a reference time Tp. The falling edge of the first demodulated signal DEM1 and the rising edge of the second demodulated signal DEM2 may be synchronized at time point tb, and the pulse of the received light RL may be divided by the pulses of the first demodulated signal DEM1 and the second demodulated signal DEM2 at time point tb. In some example embodiments, the pulse width of the first demodulated signal DEM1 may be set to be different from the pulse width of the second demodulated signal DEM2.

[0082] like Figure 5 As shown, when the time offset Ts is 0, the time point t0' corresponding to the rising edge of the emission light TL can be consistent with the time point t0 corresponding to the rising edge of the first demodulation signal DEM1. Figure 6 As shown, when the time offset Ts has a positive value, the time point t0' corresponding to the rising edge of the emission light TL may lag behind the time point t0 corresponding to the rising edge of the first demodulation signal DEM1. When the time offset Ts has a negative value, the time point t0' corresponding to the rising edge of the emission light TL may lead the time point t0 corresponding to the rising edge of the first demodulation signal DEM1.

[0083] You can do this by Figure 5 As shown in FIG, the time offset Ts is set to 0 to perform the first sampling operation. In this case, the data sampled by the first demodulation signal DEM1 corresponds to (tb-ta) and the data sampled by the second demodulation signal DEM2 corresponds to (tc-tb). Figure 6 As shown, the time offset Ts is set to a positive value to perform the second sampling operation. In this case, the data sampled by the first demodulated signal DEM1 corresponds to (tb-ta') and the data sampled by the second demodulated signal DEM2 corresponds to (tc'-tb). Therefore, for example, as shown in FIG. Figure 7 and Figure 8F As described above, the cross time offset may be determined based on a plurality of sampling data corresponding to a plurality of time offsets.

[0084] In some example embodiments, the time offset Ts may be changed by changing the delay amount (or phase) of the emission light TL and / or the delay amounts of the demodulation signal DEM1 and the demodulation signal DEM2 . Figure 2The light source 210 and / or the row scanning circuit 130 in FIG. 1 may include a delay circuit to adjust the phase of the emission light TL and / or the demodulation signals DEM1 and DEM2 based on a control signal from the controller 150. The time offset Ts can be increased by increasing the delay of the emission light TL or by decreasing the delay of the demodulation signals DEM1 and DEM2. In other words, the time offset Ts can be reduced by decreasing the delay of the emission light TL or by increasing the delay of the demodulation signals DEM1 and DEM2.

[0085] Figure 7 is a flowchart illustrating an example embodiment of a method of determining a cross-time offset, which may be used in a method of measuring a distance according to example embodiments. Figures 8A to 8F is a diagram illustrating sampling data according to a time shift of a ToF sensor according to example embodiments.

[0086] refer to Figure 7 and Figure 8A The method includes: at S310, a first straight line LN on a plane having the sampled data and the time offset as two coordinates may be determined based on the sampled data S11 and S12 of the first reference tap TA.

[0087] At S320, a second straight line LN2 on the plane may be determined based on the sampled data S21 and S22 of the second reference tap TB. The first straight line LN1 may be determined based on Formula 5 and the second straight line LN2 may be determined based on Formula 6.

[0088] S1(T)=m1*(T-Ts1)+S11 or S1(T)=m1*(T-Ts2)+S12

[0089] m1=(S12-S11) / (Ts2-Ts1) (5)

[0090] S2(T)=m2*(T-Ts2)+S21 or S2(T)=m2*(T-Ts2)+S22

[0091] m2=(S22-S21) / (Ts2-Ts1) (6)

[0092] In Formula 5 and Formula 6, S1(T) indicates the sampling data of the first reference tap TA at the time offset T, S2(T) indicates the sampling data of the second reference tap TB at the time offset T, S11 indicates the sampling data of the first reference tap TA at the first time offset Ts1, S12 indicates the sampling data of the first reference tap TA at the second time offset Ts2, S21 indicates the sampling data of the second reference tap TB at the first time offset Ts1, and S22 indicates the sampling data of the second reference tap TB at the second time offset Ts2.

[0093] like Figure 8A As shown, the slope m1 of the first straight line LN1 can be determined based on two points (Ts1, S11) and (Ts2, S12) on the plane, and the slope m2 of the second straight line LN2 can be determined based on two points (Ts1, S21) and (Ts2, S22) on the plane. In other words, the first straight line LN1 and the second straight line LN2 can be determined using data sampled according to two sampling operations corresponding to two different time offsets Ts1 and Ts2. In some example embodiments, the first straight line LN1 and the second straight line LN2 on the plane can be determined based on data sampled according to three or more sampling operations corresponding to three or more different time offsets. In this case, the formulas for the first straight line LN1 and the second straight line LN2 can be determined, for example, using a fitting technique (e.g., a least mean square method) based on three or more points on the plane.

[0094] At S330 , an intersection point CP of the first straight line LN1 and the second straight line LN2 on the plane may be determined.

[0095] At S340, the time offset corresponding to the intersection point CP on the plane may be determined as a crossing time offset Tc. The crossing time offset Tc may be determined based on Formula 7, for example, by applying the condition S1(Tc)=S2(Tc) to Formulas 5 and 6.

[0096] (m1-m2)*Tc=m1*Ts1-m2*Ts2+S21-S11 (7)

[0097] Figures 8A to 8F An example embodiment showing the value of time of flight (ToF). Figure 8A The case where ToF is 0 is shown. Figure 8B and Figure 8C The case where ToF is between 0 and Tp / 2 is shown. Figure 8D The case where ToF is Tp / 2 is shown. Figure 8E The case where ToF is between Tp / 2 and Tp is shown. Figure 8F The case where ToF is Tp is shown. Depending on ToF, the cross time offset Tc can be a positive value (e.g., Figure 8A 、 Figure 8B and Figure 8C ), 0 (for example, Figure 8D ) or negative values ​​(e.g. Figure 8E and Figure 8F In one example embodiment, the time offsets Ts1 and Ts2 may be set to negative values, such as Figure 8F shown.

[0098] The ToF may be determined using the reference time Tp and the cross time offset Tc, and the distance between the ToF sensor and the object may be determined based on the ToF expressed based on Formula 8.

[0099] ToF=2*D / c=Tp / 2–Tc (8)

[0100] Where D indicates the distance and c indicates the speed of light.

[0101] In addition, Figures 8A to 8F In , Sa indicates the reference sample data that deviates according to the characteristics of the ambient light and depth pixels. Figures 8A to 8B Regarding the first reference tap TA and the second reference tap TB, the reference sample data Sa are equal, the reference sample data may also differ according to the taps due to an offset or gain deviation of the depth pixel, and the difference in the reference sample data may affect the calculated distance.

[0102] In some example embodiments, the reference time Tp in Formula 8 may be replaced with a compensation value (e.g., corresponding to a corresponding depth pixel processed through calibration) to compensate for characteristic deviations of one or more depth pixels. The summed values ​​S11+S12 and S12+S22 of the tap signals may be substantially constant, and the offset deviation of the taps may not affect the slope m1 of the first straight line LN1 and the slope m2 of the second straight line LN2, or the change in the time offset.

[0103] In some example embodiments, the first time offset Ts1 corresponding to the first sampling operation and the second time offset Ts2 corresponding to the second sampling operation may be fixed to appropriate values. In some example embodiments, the second time offset Ts2 of the second sampling operation may be varied according to the result of the first sampling operation. In some example embodiments, the difference between the two time offsets Ts1 and Ts2 may be set to be less than Tp / 2.

[0104] Figure 9 A circuit embodiment of a depth pixel with a two-tap structure in a ToF sensor. Figure 10 It is shown that Figure 9 A timing diagram of an example embodiment of the operation of a depth pixel ToF sensor is shown. Figure 9 A two-tap configuration is shown using a separate photogate for each tap.

[0105] refer to Figure 9The depth pixel PX1 may include: a first photogate PGA and transistors TMA, TS, and TT corresponding to the first tap TA; a second photogate PGB and transistors TMB, TS, and TT corresponding to the second tap TB; transistors TRS, TSF, and TSL corresponding to the readout circuit; an overflow gate OG; and a photodiode PD. Each of the transistors TMA, TMB, TS, TT, and TRS may include a gate disposed above a semiconductor substrate and source and drain regions in the semiconductor substrate on respective sides of the gate. The gates of the transistors TMA, TMB, TS, TT, and TRS correspond to the first demodulation transfer gate TGA, the second demodulation transfer gate TGB, the storage gate SG, the floating diffusion (FD) transfer gate TG, and the reset gate RG, respectively.

[0106] A first photo gate signal SPGA is applied to the first photo gate PGA, a second photo gate signal SPGB is applied to the second photo gate PGB, a first demodulation control signal STGA is applied to the first demodulation transmission gate TGA, a second demodulation control signal STGB is applied to the second demodulation transmission gate TGB, an overflow gate voltage VOG is applied to the overflow gate OG, a storage control signal SSG is applied to the storage gate SG, an FD transmission control signal STG is applied to the FD transmission gate TG, a reset signal SRG is applied to the reset gate RG, and a selection signal SEL is applied to the gate of the selection transistor TSL. The first photo gate signal SPGA and the second photo gate signal SPGB correspond to the above-mentioned demodulation signals having different phases.

[0107] For example, as reference Figure 2 The ground is provided with demodulation signals SPGA and SPGB, demodulation transfer control signals STGA and STGB, overflow gate voltage VOG, storage control signal SSG, FD transfer control signal STG, reset signal SRG and selection signal SEL from the row scanning circuit 130 under the control of the controller 150 .

[0108] The storage gate SG is one of the charge storage structures that temporarily stores the photocharges transferred from the photodiode PD via the demodulation transfer gates TGA and TGB before transferring the photocharges to the floating diffusion area FDA and the floating diffusion area FDB. In some example embodiments, the charge storage structure may be implemented with the storage gate SG alone. In some example embodiments, the charge storage structure may be implemented with the storage gate SG and a storage diode formed in the semiconductor substrate below the storage gate SG. Using such a charge storage structure, true correlated double sampling (CDS) may be performed and noise in the readout signal may be reduced. In one example embodiment, the FD transfer gate TG and / or the storage gate SG may be omitted.

[0109] The charges stored in the floating diffusion area FDA and the floating diffusion area FDB may be provided as output signals (eg, sampled data SOA and SOB using the source follower transistor TSF and the select transistor TSL).

[0110] refer to Figure 2 、 Figure 9 and Figure 10 The light source 210 may generate emission light TL modulated according to the modulation frequency during the integration period TINT to collect photocharges generated by the incident light. The row scanning circuit 130 may apply a first photo gate signal and a second photo gate signal (e.g., a first demodulation signal SPGA and a second demodulation signal SPGB having different phases) to the first photo gate PGA corresponding to the first tap TA and the second photo gate PGB corresponding to the second tap TB, respectively.

[0111] Overflow voltage VOG applied to overflow gate OG may have a cutoff voltage level VOFF during integration period TINT to prevent photocharges from being discharged from photodiode PD. Demodulation transfer control signals STGA and STGB and storage control signal SSG are activated during integration period TINT. Therefore, photocharges collected by first demodulation signal SPGA and second demodulation signal SPGB may be stored in the semiconductor substrate below storage gate SG, respectively.

[0112] During other time periods (e.g., a reset time period TRST for initializing the depth pixel PX and a readout time period TRD for measuring the amount of photocharge collected during the integration time period TINT), the overflow gate voltage VOG may have an on-voltage level VON to discharge photocharge from the photodiode PD. The collected photocharge may be discharged to the terminal of the power supply voltage VDD during the time periods TRST and TRD other than the integration time period TINT. Therefore, the overflow gate OG may be used to implement a global shutter function.

[0113] At a first time point t10 during the readout period TRD, when the reset signal SRG is deactivated and the selection signal SEL is activated, first reset state data of the first tap TA and second reset state data of the second tap TB may be output through the column lines, respectively. At a second time point t11 ​​during the readout period TRD, when the FD transfer control signal STG is activated and the storage control signal SSG1 is deactivated, photocharges stored by the storage gate SG may be transferred to the floating diffusion area FDA and the floating diffusion area FDB, and first sampled data SOA of the first tap TA and second sampled data SOB of the second tap TB may be output through the column lines, respectively.

[0114] Figure 11A circuit embodiment of a depth pixel with a two-tap structure in a ToF sensor. Figure 12 It is shown that Figure 11 A timing diagram of an example embodiment of the operation of a depth pixel ToF sensor is shown. Figure 11 The depth pixel PX2 can be, for example, Figure 9 The depth pixel PX1 is similar.

[0115] refer to Figure 11 , the depth pixel PX2 includes a common photogate CPG instead of Figure 9 The first photogate PGA and the second photogate PGB of the depth pixel PX1. Figure 11 In the case of the depth pixel PX2, the first demodulation transmission control signal STGA and the second demodulation transmission control signal STGB correspond to the above-mentioned demodulation signals.

[0116] refer to Figure 11 and Figure 12 During the integration period TINT, the photogate voltage VPG applied to the common photogate CPG may have a DC voltage level VDC that causes the collection of photo charges, and the overflow gate voltage VOG applied to the overflow gates OG1 and OG2 may have an off-voltage level VOFF that causes the discharge of photo charges to be blocked. The DC voltage level VDC may be a voltage level between a high voltage level VH of the demodulation signals STGA and STGB and a low voltage level VL of the demodulation signals STGA and STGB.

[0117] Furthermore, during the integration period TINT, first and second demodulation signals STGA and STGB of different phases may be applied to the first and second demodulation transmission gates TGA and TGB, respectively. The phase of the first demodulation signal STGA may be synchronized with the phase of the emitted light TL. In some example embodiments, the phase difference between the first and second demodulation signals STGA and STGB may be approximately 180 degrees.

[0118] Figure 13 is a timing diagram illustrating an embodiment of operation of a pulse-based scheme to implement ToF. Figure 14 is a timing diagram illustrating an embodiment of the operation of a continuous wave based scheme of ToF.

[0119] refer to Figure 13 , Figure 2Light source 210 in the example may generate a pulse wave with a duty cycle lower than 0.5 as emitted light TL. In this case, the maximum time of flight (ToF) measurable by the ToF sensor may be limited by the period Tm of the pulses in the emitted light TL. In other words, the ToF sensor can measure distances between 0 and c / (2*fm), where c is the speed of light and fm is the modulation frequency of the emitted light TL, satisfying the formula: fm = 1 / Tm.

[0120] exist Figure 13 In this pulse scheme, a search can be performed to determine a base time offset for placing the pulse of the reception light RL in the sampling range SPL so that the reception light RL can be sampled by the demodulation signal DEM1 and the demodulation signal DEM2. Figure 15 and Figure 16 Describe the example.

[0121] refer to Figure 14 , Figure 2 Light source 210 in FIG. 2 can generate a continuous wave with a duty cycle of 0.5 as emitted light TL. In this case, the maximum ToF value measurable by the ToF sensor may be limited by the period Tm of the emitted light TL. Here, period Tm can satisfy the formula: Tm = 2*Tp. In other words, the ToF sensor can measure the distance between 0 and c*Tp, where c is the speed of light and Tp is the reference time corresponding to the pulse width of the emitted light TL.

[0122] Figure 15 and Figure 16 : is a diagram showing an embodiment of a scanning method based on time shift of a ToF sensor. Figure 15 , the first sampled data corresponding to the first tap TA and the second sampled data corresponding to the second tap TB are shown, and they can be sampled according to the time offset. In the region where the time offset is less than Tsa, the first sampled data and the second sampled data maintain the value corresponding to the reference sampled data Sa. In the region where the time offset is between Tsa and Tsb, the first sampled data increases and the second sampled data maintains the value corresponding to the reference sampled data Sa. In the region where the time offset is between Tsb and Tsc, the first sampled data decreases and the second sampled data increases. In the region where the time offset is between Tsc and Tsd, the first sampled data maintains the value corresponding to the reference sampled data Sa and the second sampled data decreases. In the region where the time offset is greater than Tsd, the first sampled data and the second sampled data maintain the value corresponding to the reference sampled data Sa.

[0123] Therefore, the region of the time offset between Tsb and Tsc corresponds to the valid region VAL where the cross point CP may exist. Therefore, a base time offset for placing the pulse of the received light RL in the sampling range SPL can be searched so that the received light RL can be sampled by the demodulation signal DEM1 and the demodulation signal DEM2.

[0124] Figure 16 An example case is shown where the cycle time period Tm of the emitted light TL is 9Tp and the time of flight (ToF) is between 6Tp and 7Tp. The first case CS1 corresponds to a base time offset of 0 and the second case CS2 corresponds to a base time offset of Tb. Figure 15 The second case CS2 corresponds to a region where the time offset in the received light RL is less than Tsa, so that the pulse of the received light RL is not included in the sampling range SPL. In contrast, the second case CS2 corresponds to a region where the effective region VAL (eg, Figure 15 The time shift in the sampling range SPL corresponds to the region between Tsb and Tsc) so that the pulse of the received light RL is included in the sampling range SPL.

[0125] In some example embodiments, to search for the base time offset Tb, the sampling operation may be repeatedly performed by varying the time offset between the emission light TL and the demodulation signals DEM1 and DEM2 by Tp or Tp / 2 until the valid area VAL is found. Thus, the base time offset Tb between the emission light TL and the demodulation signals DEM1 and DEM2 may be searched for and determined.

[0126] After determining the base time offset Tb, the cross time offset can be determined by performing a first sampling operation corresponding to the first time offset Tb+Ts1 and a second sampling operation corresponding to the second time offset Tb+Ts2 as described above. To include the case of applying the base time offset Tb, Formula 8 can be generalized as the following Formula 9.

[0127] ToF=2D / c=Tb+(Tp / 2–Tc) (9)

[0128] Figure 17 and Figure 18 is a diagram illustrating an example embodiment of a sampling operation of a ToF sensor. Figure 17 and Figure 18 , the time offset between the demodulation signal DEM1 and the demodulation signal DEM2 can be changed during a single integration period TINT to collect photo charges generated by the incident light.

[0129] In some example embodiments, the integration time period TINT may be divided into a plurality of sub-time periods and different time offsets may be applied to the sub-time periods. Figure 17As shown, the integration period TINT can be divided into first to third sub-periods TSUB1 to TSUB3. A time offset Ts-Δ can be applied in the first sub-period TSUB1, a time offset Ts can be applied in the second sub-period TSUB2, and a time offset Ts+Δ can be applied in the third sub-period TSUB3. After the integration period TINT, the sampled data can be read out as described above.

[0130] Figure 18 Shown by applying Figure 17 An example of time-shifted sampled data obtained by the method. Figure 18 As shown, the cross time offset Tc corresponding to the cross point CP can be determined based on the sampling data S11 and S12 obtained by the first sampling operation using multiple time offsets Ts1-Δ, Ts1 and Ts1+Δ and based on the sampling data S21 and S22 obtained by the second sampling operation using multiple time offsets Ts1-Δ, Ts1 and Ts1+Δ.

[0131] In some examples, there may be a deviation in the sampled data of each sampling operation, so the accuracy of the cross time offset Tc can increase with the increase of the number of samples. However, the operating speed of the ToF sensor decreases with the increase of the number of samples. Figure 17 and Figure 18 As described in the embodiment of the present invention, the time offset can be changed in a single integration period TINT and the average sampled data of multiple time offsets can be read out by a single read operation. Therefore, the accuracy of distance measurement can be improved by eliminating one or more characteristic deviations (e.g., fixed pattern noise, wobble noise, etc.) without reducing the operating speed of the ToF sensor.

[0132] Figure 19 and Figure 20 is a diagram illustrating an example embodiment of a time offset for distance measurement by a ToF sensor. Figure 19 and Figure 20 , in these embodiments, Figure 2 The controller 150 in the embodiment can control the row scanning circuit 130 to change the time offset between the emission light TL and the demodulation signals DEM1 and DEM2. This can be achieved by changing the pulse width or duty cycle of the demodulation signals DEM1 and DEM2.

[0133] exist Figure 19In the example, the first case CS1 corresponds to a case where the pulse width of the first demodulated signal DEM1 is equal to the pulse width of the second demodulated signal DEM2. The second case CS2 corresponds to a case where the pulse width of the first demodulated signal DEM1 is smaller than the pulse width of the second demodulated signal DEM2. The third case CS3 corresponds to a case where the pulse width of the first demodulated signal DEM1 is larger than the pulse width of the second demodulated signal DEM2. The area of ​​the hatched portion is proportional to the corresponding sampled data. The pulse width of the received light RL can be divided by the first demodulated signal DEM1 and the second demodulated signal DEM2 at time point tb1 in the first case CS1, at time point tb2 in the second case CS2, and at time point tb3 in the third case CS3.

[0134] like Figure 20 As shown, time point tb1 of the first case CS1 corresponds to time offset Ts, time point tb2 of the second case CS2 corresponds to time offset Ts-Δ, and time point tb3 of the third case CS3 corresponds to time offset Ts+Δ. Therefore, by changing the pulse width or duty cycle of the demodulated signals DEM1 and DEM2, sampled data S11-S13 and S21-S23 can be obtained, which is substantially the same as the result when the time offset is changed. Therefore, the time offset can be reduced by reducing the pulse width of the first demodulated signal DEM1 applied to the first tap TA and increasing the pulse width of the second demodulated signal DEM2 applied to the second tap TB.

[0135] Figure 21 is a flowchart illustrating a method of measuring a distance according to an example embodiment, Figure 22 and Figure 23 is used to describe Figure 21 An example of a graph of the method.

[0136] refer to Figure 21 The approach speed of the object may be determined based on a plurality of sampling data corresponding to a plurality of time offsets (S510). The plurality of sampling data may be corrected based on the approach speed (S520) and the cross time offset may be corrected based on the corrected plurality of sampling data (S530).

[0137] exist Figure 22In

[10] , P0 indicates the position of the ToF sensor, P1 indicates the position of the object at time t1 when a first sampling operation corresponding to a first time offset Ts1 is performed, and P2 indicates the position of the object at time t2 when a second sampling operation corresponding to a second time offset Ts2 is performed. For example, when an object approaches the ToF sensor, the distance D2 at time t2 is smaller than the distance D1 at time t1. When the object is moving, the sampling data may vary depending on the distance between the ToF sensor and the object, and the intersection time offset Tc corresponding to the intersection point CP may be affected by the varying sampling data. In other words, the original intersection time offset Tc (corresponding to the original intersection point CP of straight lines LN1 and LN2 based on the sampling data S11 and S21 corresponding to the first time offset Ts1 and the sampling data S12 and S22 corresponding to the second time offset Ts2) may not reflect the accurate time of flight. The intensity of the received light RL reflected by the object may be inversely proportional to the square of the distance, as shown in Equation 10.

[0138] S11+S21=k / D1 2

[0139] S12+S22=k / D2 2 (10)

[0140] In Formula 10, S11+S21 corresponds to the sum of the sampled data at time point t1, S12+S22 corresponds to the sum of the sampled data at time point t2, and k indicates a constant depending on, for example, the medium through which light propagates, the reflection coefficient of the object, etc. The distance difference D1-D2 between time point t1 and time point t2 can be obtained based on Formula 10, and the approaching speed of the object can be obtained based on Formula 11.

[0141] Va=(D1-D2) / (t1-t2) (11)

[0142] Figure 23 An example is shown including first corrected sampled data S11' and S21'. These first corrected sampled data S11' and S21' are obtained by correcting the sampled data S11 and S21 corresponding to the first time offset Ts1, so that the corrected sampled data S11' and S21' can be normalized to the sampled data S12 and S22 corresponding to the second time offset Ts2. In this case, the second corrected sampled data S12' and S22' can be equal to the sampled data S12 and S22. Corrected straight lines LN1' and LN2' and a corrected intersection point CP' can be determined based on the first corrected sampled data S11' and S12' and the second corrected sampled data S21' and S22'. The original intersection time offset Tc based on the original sampled data S11, S12, S21, and S22 can be corrected to the corrected intersection time offset Tc'.

[0143] Figure 24 This is a circuit embodiment of a depth pixel PX3 with a four-tap structure in a ToF sensor. Figure 25 It is shown that Figure 24 Figure 1 shows a timing diagram of an example operation of a ToF sensor with a depth pixel. Figure 24 A four-tap configuration is shown using a separate photogate for each tap.

[0144] refer to Figure 24 The depth pixel PX3 may include: a first photogate PGA and transistors TMA, TS1, and TT1 corresponding to a first tap TA; a second photogate PGB and transistors TMB, TS1, and TT1 corresponding to a second tap TB; a third photogate PGC and transistors TMC, TS2, and TT2 corresponding to a third tap TC; a fourth photogate PGD and transistors TMD, TS2, and TT2 corresponding to a fourth tap TD; transistors TRS1, TRS2, TSF1, TSF2, TSL1, and TSL2 corresponding to a readout circuit; and an overflow gate OG and a photodiode corresponding to a shared circuit. Each of the transistors TMA, TMB, TMC, TMD, TS1, TS2, TT1, TT2, TRS1, and TRS2 may include a gate disposed above a semiconductor substrate and source and drain regions disposed in the semiconductor substrate on respective sides of the gate. The gates of transistors TMA, TMB, TMC, TMD, TS1, TS2, TT1, TT2, TRS1 and TRS2 can correspond to the first demodulation transmission gate TGA, the second demodulation transmission gate TGB, the third demodulation transmission gate TGC, the fourth demodulation transmission gate TGD, the storage gate SG, the FD transmission gate TG, and the reset gates RG1 and RG2, respectively.

[0145] First to fourth photo gate signals SPGA to SPGD are applied to first to fourth photo gates PGA to PGD, overflow gate voltage VOG is applied to overflow gate OG, storage control signals SSG1 and SSG2 are applied to storage gates SG1 and SG2, demodulation transfer control signals STGA to STGD are applied to demodulation transfer gates TGA to TGD, FD transfer control signals STG1 and STG2 are applied to FD transfer gates TG1 and TG2, reset signals SRG1 and SRG2 are applied to reset gates RG1 and RG2, and select signals SEL1 and SEL2 are applied to gates of select transistors TSL1 and TSL2. The first to fourth photo gate signals SPGA to SPGD correspond to the above-mentioned demodulation signals having different phases.

[0146] You can refer to the above Figure 2The ground provides photo gate signals SPGA~SPGD, overflow gate voltage VOG, demodulation transfer control signals STGA~STGD, storage control signals SSG1 and SSG2, FD transfer control signals STG1 and STG2, reset signals SRG1 and SRG2, and selection signals SEL1 and SEL2 from the row scanning circuit 130 under the control of the controller 150.

[0147] refer to Figure 2 、 Figure 24 and Figure 25 In the distance detection mode, the light source 210 can generate emission light TL modulated according to the modulation frequency during the integration period TINT to collect photocharges generated by the incident light. The row scanning circuit 130 can apply first to fourth sampling control signals (e.g., first to fourth photo gate signals SPGA to SPGD having different phases) to the first to fourth photo gates PGA to PGD corresponding to the first to fourth taps TA to TD.

[0148] Overflow voltage VOG applied to overflow gate OG may have a cutoff voltage level VOFF to prevent photocharges from being discharged from photodiode PD during integration period TINT. Demodulation transfer control signals STGA-STGD and storage control signals SSG1 and SSG2 are activated during integration period TINT. Therefore, photocharges collected by first to fourth photo gate signals SPGA-SPGD may be stored in the semiconductor substrate beneath storage gates SG1 and SG2, respectively.

[0149] During other time periods (e.g., a reset time period TRST for initializing the depth pixel PX1 and a readout time period TRD for measuring the amount of photocharge collected during the integration time period TINT), the overflow gate voltage VOG may have an on-voltage level VON to discharge photocharge from the photodiode PD. The collected photocharge may be discharged to the terminal of the power supply voltage VDD during the time periods TRST and TRD other than the integration time period TINT. Thus, a global shutter function may be implemented using the overflow gate OG.

[0150] At a first time point t10 during the readout period TRD, when the reset signal SRG1 is deactivated and the selection signal SEL1 is activated, first reset state data of the first tap TA and second reset state data of the second tap TB may be output through the column lines COL1 and COL2, respectively. At a second time point t11 ​​during the readout period TRD, when the FD transfer control signal STG1 is activated and the storage control signal SSG1 is deactivated, photocharges stored by the storage gate SG may be transferred to the floating diffusion area FDA and the floating diffusion area FDB, and first sampled data SOA of the first tap TA and second sampled data SOB of the second tap TB may be output through the column lines COL1 and COL2, respectively.

[0151] At a third time point t20 of the read period TRD, when the reset signal SRG2 is deactivated and the selection signal SEL2 is activated, the third reset state data of the third tap TC and the fourth reset state data of the fourth tap TD may be output through the column lines COL2 and COL1, respectively. At a fourth time point t21 during the read period TRD, when the FD transfer control signal STG2 is activated and the storage control signal SSG2 is deactivated, the photocharges stored by the storage gate SG may be transferred to the floating diffusion areas FDC and FDD, and the third sampled data SOC of the third tap TC and the fourth sampled data SOD of the fourth tap TD may be output through the column lines COL2 and COL1, respectively.

[0152] Figure 26 A circuit embodiment of a depth pixel with a four-tap structure in a ToF sensor. Figure 27 It is shown that Figure 26 Figure 1 shows a timing diagram of an example operation of a ToF sensor with a depth pixel. Figure 26 The depth pixel PX4 can be similar to Figure 24 The depth pixel PX3.

[0153] refer to Figure 26 , the depth pixel PX4 includes a common photogate CPG (instead of Figure 24 The first to fourth photogates PGA-PGD of the depth pixel PX3 are Figure 26 In the case of the depth pixel PX4, the first to fourth demodulation transmission control signals STGA~STGD may correspond to the above-mentioned demodulation signals.

[0154] refer to Figure 26 and Figure 27During the integration period TINT, the photogate voltage VPG applied to the common photogate CPG may have a DC voltage level VDC that causes the collection of photocharges. The overflow gate voltage VOG applied to the overflow gates OG1 and OG2 may have a cutoff voltage level VOFF that prevents the discharge of photocharges. The DC voltage level VDC may be a voltage level between a first predetermined (e.g., high) voltage level VH of the demodulation signals STGA-STGD and a second predetermined (e.g., low) voltage level VL of the demodulation signals STGA-STGD.

[0155] In addition, during the integration period TINT, the first to fourth demodulation signals STGA to STGD of different phases may be applied to the first to fourth demodulation transmission gates TGA to TGD, respectively. In an example embodiment, the phase difference between the first demodulation signal STGA and the second demodulation signal STGB may be approximately 90 degrees, the phase difference between the first demodulation signal STGA and the third demodulation signal STGC may be approximately 180 degrees, and the phase difference between the first demodulation signal STGA and the fourth demodulation signal STGD may be approximately 270 degrees.

[0156] Figure 28 and Figure 29 is a diagram illustrating a time offset of a ToF sensor according to an example embodiment. Figure 28 and Figure 29 , the following contents regarding the four-tap structure are shown: the transmitted light TL, the received light RL corresponding to an arbitrary time of flight ToF, and the timings of the first to fourth demodulated signals DEM1 to DEM4 corresponding to the first to fourth taps. Figure 28 Shows the case where the time offset Ts is 0 and Figure 29 The case where the time offset Ts has a positive value is shown.

[0157] refer to Figure 28 and Figure 29 , the pulse width of the emission light TL and the pulse widths of the plurality of demodulation signals may be substantially equal to each other as a reference time Tp. The falling edge of the first demodulation signal DEM1 and the rising edge of the second demodulation signal DEM2 may be synchronized at time point tb1, the falling edge of the second demodulation signal DEM2 and the rising edge of the third demodulation signal DEM3 may be synchronized at time point tb2, and the falling edge of the third demodulation signal DEM3 and the rising edge of the fourth demodulation signal DEM4 may be synchronized at time point tb3.

[0158] The pulse of the reception light RL may be divided at the time point tb1 by the pulses of the first demodulation signal DEM1 and the second demodulation signal DEM2, for example, as Figure 28 and Figure 29As shown. In this case, the first tap may correspond to the first reference tap and the second tap may correspond to the second reference tap. Depending on the ToF, the pulse of the received light RL may be divided by the pulses of the second demodulation signal DEM2 and the third demodulation signal DEM3 at time point tb2, or the pulse of the received light RL may be divided by the pulses of the third demodulation signal DEM3 and the fourth demodulation signal DEM4 at time point tb3. In some example embodiments, the pulse widths of the first to fourth demodulation signals DEM1 to DEM4 may be set to be different from each other.

[0159] like Figure 28 As shown, when the time offset Ts is 0, the time point t0' corresponding to the rising edge of the emission light TL can be consistent with the time point t0 corresponding to the rising edge of the first demodulation signal DEM1. Figure 29 As shown, when the time offset Ts has a positive value, the time point t0' corresponding to the rising edge of the emission light TL may lag behind the time point t0 corresponding to the rising edge of the first demodulation signal DEM1. When the time offset Ts has a negative value, the time point t0' corresponding to the rising edge of the emission light TL may lead the time point t0 corresponding to the rising edge of the first demodulation signal DEM1.

[0160] You can do this by Figure 28 As shown in FIG, the time offset Ts is set to 0 to perform the first sampling operation. In this case, the data sampled by the first demodulation signal DEM1 corresponds to (tb1-ta) and the data sampled by the second demodulation signal DEM2 corresponds to (tc-tb1). Figure 29 As shown, the time offset Ts is set to a positive value to perform the second sampling operation. In this case, the data sampled by the first demodulated signal DEM1 corresponds to (tb1-ta') and the data sampled by the second demodulated signal DEM2 corresponds to (tc'-tb1). Therefore, the cross time offset can be determined based on a plurality of sampled data corresponding to a plurality of time offsets, for example, as shown in FIG. Figure 7 and Figure 8F As stated.

[0161] In some example embodiments, the time offset Ts may be changed by changing the delay amount (or phase) of the emission light TL and / or the delay amount of the demodulation signals DEM1 ˜ DEM4 . Figure 2The light source 210 and / or the row scanning circuit 130 in the embodiment may include or be coupled to a delay circuit to adjust the phase of the emission light TL and / or the demodulation signals DEM1-DEM4 based on a control signal from the controller 150. The time offset Ts can be increased by increasing the delay of the emission light TL or by decreasing the delay of the demodulation signals DEM1-DEM4. In other words, the time offset Ts can be reduced by decreasing the delay of the emission light TL or by increasing the delay of the demodulation signals DEM1-DEM4.

[0162] Figure 30 is a diagram illustrating sampling data according to a time shift of a ToF sensor according to an example embodiment. Figure 30 , when ToF is between 0 and Tp, the crossing time offset Tc1 corresponding to the crossing point CP1 can be determined based on the sampling data corresponding to the time offsets Ts1 and Ts2 of the first tap TA and the second tap TB. In other words, when ToF is between 0 and Tp, the first tap TA corresponds to the first reference tap and the second tap TB corresponds to the second reference tap.

[0163] When the ToF is between Tp and 2Tp, the intersection time offset Tc2 corresponding to the intersection point CP2 can be determined based on the sampling data corresponding to the time offsets Tp+Ts1 and Tp+Ts2 of the second tap TB and the third tap TC. In other words, when the ToF is between Tp and 2Tp, the second tap TB corresponds to the first reference tap and the third tap TC corresponds to the second reference tap.

[0164] When the ToF is between 2Tp and 3Tp, the intersection time offset Tc3 corresponding to the intersection point CP3 can be determined based on the sampling data corresponding to the time offsets 2Tp+Ts1 and 2Tp+Ts2 of the third tap TC and the fourth tap TD. In other words, when the ToF is between 2Tp and 3Tp, the third tap TC corresponds to the first reference tap and the fourth tap TD corresponds to the second reference tap.

[0165] Therefore, when the depth pixel has a four-tap structure including a first tap to a fourth tap, one of the first to fourth taps TA~TD can be determined as a first reference tap and another one of the first to fourth taps TA~TD can be determined as a second reference tap according to the distance between the ToF sensor and the object.

[0166] When the depth pixel has a four-tap structure, the sampling data of the first reference tap and the second reference tap can be corrected based on the sampling data of two taps other than the first reference tap and the second reference tap among the first to fourth taps TA to TD. Figure 30When the ToF is between Tp and 2Tp, the second tap TB corresponds to the first reference tap and the third tap TC corresponds to the second reference tap. In this case, the sampled data of the first tap TA and the fourth tap TD correspond to the reference sampled data Sa. Corrected sampled data can be obtained by subtracting the reference sampled data Sa of the first tap TA and the fourth tap TD from the sampled data of the second tap TB and the third tap TC corresponding to the first reference tap and the second reference tap. The accuracy of distance measurement can therefore be further improved based on the corrected sampled data.

[0167] Figure 31 is a diagram illustrating a scanning method of a base time offset of a ToF sensor according to an example embodiment. Figure 31 , first to fourth sample data corresponding to the first to fourth taps TA to TD according to time offset sampling are respectively shown.

[0168] In a region where the time offset is smaller than Tsa, the first to fourth sampling data may maintain values ​​corresponding to the reference sampling data Sa.

[0169] In a region where the time offset is between Tsa and Tsb, the first sampling data may increase and the second to third sampling data may maintain values ​​corresponding to the reference sampling data Sa.

[0170] In a region where the time offset is between Tsb and Tsc, the first sample data may decrease, the second sample data may increase, and the third sample data and the fourth sample data may maintain values ​​corresponding to the reference sample data Sa.

[0171] In a region where the time offset is between Tsc and Tsd, the second sample data may decrease, the third sample data may increase, and the first sample data and the fourth sample data may maintain values ​​corresponding to the reference sample data Sa.

[0172] In a region where the time offset is between Tsd and Tse, the third sampling data may decrease, the fourth sampling data may increase, and the first sampling data and the second sampling data may maintain values ​​corresponding to the reference sampling data Sa.

[0173] In a region where the time offset is between Tse and Tsf, the first to third sampling data may maintain values ​​corresponding to the reference sampling data Sa and the fourth sampling data may decrease.

[0174] In a region where the time shift is greater than Tsf, the first to fourth sampling data maintain values ​​corresponding to the reference sampling data Sa.

[0175] Therefore, the region when the time offset is between Tsb and Tse may correspond to the valid region VAL where the cross point CP may exist. Figure 15 and Figure 16 In the embodiment, the basic time offset may be searched to place the pulse of the reception light RL in the sampling range SPL, so that the reception light RL may be sampled by the demodulation signal DEM1 and the demodulation signal DEM2.

[0176] Figure 32 is a block diagram illustrating a computing system 1000 including a ToF sensor according to example embodiments.

[0177] refer to Figure 32 , the computing system 1000 may include a processor 1010, a memory device 1020, a storage device 1030, an input / output (I / O) device 1040, a power supply 1050, and / or a ToF sensor 100. In an embodiment, the computing system 1000 may also include a port for communicating with electronic devices such as, for example, a video card, a sound card, a memory card, a USB device, or the like.

[0178] The processor 1010 can perform specific calculations and / or tasks, and for example, the processor 1010 can be a microprocessor, a central processing unit (CPU), a digital signal processor, or another type of processing structure. The processor 1010 can communicate with the memory device 1020, the storage device 1030, and the input / output device 1040 via an address bus, a control bus, and / or a data bus. The processor 1010 can be coupled to an expansion bus such as a peripheral component interconnect (PCI) bus. The memory device 1020 can store data used to operate the computing system 1000.

[0179] The memory device 1020 can be implemented as, for example, dynamic random access memory (DRAM), mobile DRAM, static random access memory (SRAM), phase change random access memory (PRAM), resistive random access memory (RRAM), nano-floating gate memory (NFGM), polymer random access memory (PoRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), etc.

[0180] The storage device 1030 may include, for example, a solid-state drive, a hard disk drive, a CD-ROM, etc. The input / output device 1040 may include, for example, an input device such as a keyboard, a mouse, a keypad, etc. and an output device such as, for example, a printer, a display device, etc. The power supply 1050 may provide power to the computing device 1000.

[0181] The ToF sensor 100 may be coupled to the processor 1010 via one or more buses and / or one or more other communication links. The ToF sensor 100 may be integrated into the same chip as the processor 1010, or the ToF sensor 100 and the processor 1010 may be integrated into separate chips. According to the embodiments described herein, the ToF sensor 100 may use cross-time offsets determined based on sampled data corresponding to different time offsets to improve the accuracy of distance measurement.

[0182] According to one embodiment, an apparatus comprises a memory area and at least one processor. The apparatus may correspond to a time-of-flight sensor or one or more elements of a time-of-flight sensor. The one or more elements may comprise, for example Figure 1 The controller 150 shown or one or more other elements of the embodiments described herein. The at least one processor may include the controller 150, the sensing unit, and / or a processor coupled to the time-of-flight sensor. When the at least one processor includes the controller 150, the storage area may be connected to, for example, Figure 1 The illustrated storage area 151 corresponds to or may be a storage area at another location.

[0183] The storage area is configured to store instructions to be executed by at least one processor. The storage area can be a non-transitory computer-readable medium, such as a volatile or non-volatile memory. When executing the instructions, the at least one processor can perform operations corresponding to any of the embodiments described herein. For example, in one embodiment, the at least one processor can execute instructions to generate a time offset between the emitted light and the demodulated signal, generate sampled data corresponding to the time offset, determine the cross-time offset based on the sampled data of the first reference tap being substantially equal to the sampled data of the second reference tap, and determine the distance between the time-of-flight sensor and the object based on the cross-time offset. Emitted light can be output from a time-of-flight sensor including a first reference tap and a second reference tap, and the object can be illuminated as described herein. In addition, as described in accordance with the aforementioned embodiments, the demodulated signal and the sampled data can be derived based on the emitted light (e.g., received light) reflected from the object. As previously described, the demodulated signal can have different phases.

[0184] Method described herein, process and / or operation can be performed by the code or instruction to be performed by computer, processor, controller or other signal processing equipment.Computer, processor, controller or other signal processing equipment can be element described herein or the element except element described herein.Because the algorithm of the basis of formation method (or the operation of computer, processor, controller or other signal processing equipment) is described in detail, the code or instruction for realizing the operation of method embodiment can convert computer, processor, controller or other signal processing equipment into the special purpose processor for performing the method herein.

[0185] Furthermore, another embodiment may include a computer-readable medium, such as a non-transitory computer-readable medium, for storing the above-mentioned codes or instructions. The computer-readable medium may be a volatile or non-volatile memory or other storage device, which may be removably or fixedly coupled to a computer, processor, controller, or other signal processing device for executing the operations of the apparatus embodiments or the codes or instructions of the method embodiments described herein.

[0186] The controllers, processors, devices, modules, units, converters, devices, sensors, multiplexers, generators, logic, interfaces, decoders, drivers, generators, and other signal generation and signal processing features of the embodiments described herein may be implemented, for example, in non-transitory logic that may include hardware, software, or both. When implemented at least partially in hardware, the controllers, processors, devices, modules, units, converters, devices, sensors, multiplexers, generators, logic, interfaces, decoders, drivers, generators, and other signal generation and signal processing features may be, for example, any of a variety of integrated circuits, including but not limited to an application specific integrated circuit, a field programmable gate array, a combination of logic gates, a system on a chip, a microprocessor, or another type of processing or control circuit.

[0187] When being realized at least in part in software, controller, processor, equipment, module, unit, converter, device, sensor, multiplexer, generator, logic, interface, decoder, driver, generator and other signal generation and signal processing features can comprise for example for storing the code to be performed by for example computer, processor, microprocessor, controller or other signal processing equipment or the memory or other storage device of instruction.Computer, processor, microprocessor, controller or other signal processing equipment can be element described herein or the element except element described herein.Because the algorithm of the basis of formation method (or the operation of computer, processor, microprocessor, controller or other signal processing equipment) is described in detail, the code or instruction for realizing the operation of method embodiment can convert computer, processor, controller or other signal processing equipment into the special-purpose processor for performing method described herein.

[0188] Embodiments of the present inventive concept can be applied to any electronic device and system. Examples include, but are not limited to, memory cards, solid-state drives (SSDs), embedded multimedia cards (eMMCs), universal flash memory (UFS), mobile phones, smartphones, personal digital assistants (PDAs), portable multimedia players (PMPs), digital cameras, video recorders, personal computers (PCs), server computers, workstations, laptop computers, digital TVs, set-top boxes, portable game consoles, navigation systems, wearable devices, Internet of Things (IoT) devices, Internet of Everything (IoE) devices, e-books, virtual reality (VR) devices, augmented reality (AR) devices, navigation systems for vehicles, monitoring systems, autofocus systems, tracking systems, motion detection systems, and the like.

[0189] The foregoing is illustrative of example embodiments and should not be construed as limiting thereof. Although a few example embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible without materially departing from the example embodiments of the present invention.

Claims

1. A method for measuring distance using a time-of-flight (ToF) sensor, the ToF sensor comprising at least one depth pixel and a light source for irradiating an object with emission light, the at least one depth pixel having a multi-tap structure including multiple taps to generate a plurality of sampled data based on received light and a plurality of demodulated signals having different phases, the emission light being reflected from the object and returned to the ToF sensor as the received light, the method comprising: generating a plurality of time offsets between the emitted light and a plurality of demodulated signals; performing a plurality of sampling operations to generate the plurality of sampling data corresponding to the plurality of time offsets; determining a cross time offset based on the plurality of sampling data corresponding to the plurality of time offsets so that sampling data of a first reference tap and a second reference tap among the plurality of taps become equal to each other with respect to the cross time offset; as well as A distance between the ToF sensor and the object is determined based on the cross-time offset.

2. The method according to claim 1, wherein Generating the plurality of time offsets comprises: Changing the delay of the emitted light, or The delay amounts of the plurality of demodulated signals are changed.

3. The method according to claim 1, wherein Generating the plurality of time offsets comprises: The pulse widths of the plurality of demodulated signals are changed.

4. The method according to claim 1, wherein Determining the crossover time offset includes: determining, based on the sampled data of the first reference tap, a first straight line on a plane having the sampled data and the time offset information as two coordinates; determining a second straight line on the plane based on the sampled data of the second reference tap; determining an intersection point of the first straight line and the second straight line on the plane; and The crossing time offset corresponding to the crossing point on the plane is determined.

5. The method according to claim 1, further comprising: A base time offset is searched for to place the pulse of the received light within a sampling range so that the received light is sampled by the plurality of demodulated signals.

6. The method according to claim 1, further comprising: The time offset between the demodulated signals is varied during a single integration period to collect the photocharges generated by the incident light.

7. The method according to claim 1, further comprising: An approaching speed of the object is determined based on the plurality of sampling data corresponding to the plurality of time offsets.

8. The method according to claim 7, further comprising: correcting the plurality of sampling data based on the approaching speed; as well as The cross time offset is corrected based on the corrected plurality of sampling data.

9. The method according to claim 1, wherein: The depth pixel has a two-tap structure including a first tap and a second tap, and The first tap corresponds to the first reference tap and the second tap corresponds to the second reference tap, regardless of a distance between the ToF sensor and the object.

10. The method of claim 1, wherein: The depth pixel has a four-tap structure including first to fourth taps, and According to the distance between the ToF sensor and the object, one of the first to fourth taps is determined as the first reference tap and the other one of the first to fourth taps is determined as the second reference tap.

11. The method according to claim 10, further comprising: The sampling data of the first reference tap and the second reference tap are corrected based on the sampling data of two taps other than the first reference tap and the second reference tap among the first to fourth taps.

12. The method according to claim 1, wherein The pulse width of the emission light and the pulse widths of the plurality of demodulation signals are equal to each other as a reference time.

13. The method according to claim 12, wherein: Generating the plurality of time offsets comprises: A difference between two time offsets among the plurality of time offsets is set to be smaller than half of the reference time.

14. The method of claim 1, wherein: The emitted light is a pulse wave having a duty cycle less than 0.5, and The method includes measuring, by the ToF sensor, a distance between 0 and c / (2*fm), where c is the speed of light and fm is a modulation frequency of the emitted light.

15. The method of claim 1, wherein: The emitted light is a continuous wave with a duty cycle of 0.5, and The method includes measuring, by the ToF sensor, a distance between 0 and c*Tp, where c is the speed of light and Tp is a reference time corresponding to a pulse width of the emitted light.

16. A method for measuring distance using a time-of-flight (ToF) sensor, the ToF sensor comprising at least one depth pixel and a light source for irradiating an object with emission light, the depth pixel having a multi-tap structure including multiple taps to generate a plurality of sampled data based on received light and a plurality of demodulated signals having different phases, the emission light being reflected from the object and returned to the ToF sensor as the received light, the method comprising: generating a first time offset between the emitted light and the plurality of demodulated signals; performing a first sampling operation to generate first sampling data corresponding to the first time offset; generating a second time offset between the emitted light and the plurality of demodulated signals; performing a second sampling operation to generate second sampled data corresponding to the second time offset; determining a cross time offset based on the first sampling data and the second sampling data so that sampling data of a first reference tap and a second reference tap among the plurality of taps become equal to each other with respect to the cross time offset; as well as A distance between the ToF sensor and the object is determined based on the cross-time offset.

17. The method according to claim 16, wherein Determining the crossover time offset includes: determining a first straight line on a plane having the sampled data and the time offset as two coordinates based on the first sampled data and the second sampled data of the first reference tap; determining a second straight line on the plane based on the first sampled data and the second sampled data of the second reference tap; determining an intersection point of the first straight line and the second straight line on the plane; and The crossing time offset corresponding to the crossing point on the plane is determined.

18. A time-of-flight (ToF) sensor, comprising: a light source configured to illuminate an object with emitted light; A sensor comprising a pixel array and a row scanning circuit, wherein the pixel array comprises one or more depth pixels, each of the one or more depth pixels having a multi-tap structure including a plurality of taps and configured to generate a plurality of sampling data based on received light and a plurality of demodulation signals having different phases, wherein the emitted light is reflected from the object back to the ToF sensor as the received light, and the row scanning circuit is configured to generate the plurality of demodulation signals applied to the plurality of taps; as well as a controller configured to perform a plurality of sampling operations by applying the plurality of demodulation signals having a plurality of time offsets relative to the emission light, so as to generate the plurality of sampling data corresponding to the plurality of time offsets, wherein the controller is configured to: determining a cross time offset based on the plurality of sample data corresponding to the plurality of time offsets so that sample data of a first reference tap and a second reference tap among the plurality of taps become equal to each other with respect to the cross time offset, and A distance between the ToF sensor and the object is determined based on the cross-time offset.

19. The ToF sensor according to claim 18, wherein: The controller is configured to control the light source to change an amount of delay of the emitted light to generate the plurality of time offsets.

20. The ToF sensor according to claim 18, wherein The controller is configured to control the row scanning circuit to change at least one of a delay amount or a pulse width of the plurality of demodulated signals.

Images