Method for measuring distance using time-of-flight method and system for measuring distance

Abstract

A method for measuring distance using time of flight, performing a measurement cycle includes: a first step, performed once or multiple times, the first step including: a first sub-step for emitting pulsed light from at least one light source; a second sub-step for sequentially activating each tap of the photodetector including a plurality of taps within a predetermined time period to store a signal generated by the photodetector receiving light, wherein the plurality of taps are sequentially activated so that the completion of activation of the last tap is offset from the emission of the pulsed light by a predetermined offset time greater than or equal to zero, a second step for outputting the stored signal from the tap and outputting a differential signal of the output signal, wherein the measurement cycle is repeated as the offset time increases until the reflected light returns to the photodetector during the activation period of any one of the plurality of taps and the differential signal becomes a predetermined value greater than zero, wherein the distance from the photodetector to the object is measured based on the offset time of the measurement cycle.

Description

Technical Field

[0001] The present invention relates to a method for measuring distance using a time-of-flight method and a system for measuring distance. Background Art

[0002] Recently, the demand for three-dimensional sensing has been increasing. Mobile devices including depth imaging cameras have also been proposed. Conventionally, a multi-camera parallax system is known as a method for achieving three-dimensional sensing and depth imaging. However, the multi-camera parallax system is generally large and complicated to operate. The depth measurement accuracy of such a system is insufficient. Therefore, a depth sensor using infrared light is recently used instead of the multi-camera parallax system.

[0003] Time-of-flight (ToF) systems are known as depth sensors, which use infrared light. ToF systems are generally divided into two categories: direct ToF (dToF) systems and indirect ToF (iToF) systems. Both types of systems detect the flight time between the emission of light and the return of light reflected on an object.

[0004] like Fig.10 As shown, a conventional dToF system directly detects the flight time of light using a time counter 108. A dToF system generally includes at least one dToF sensor 102. The dToF sensor 102 includes a photodetector, an amplifier for amplifying a signal output from the photodetector in response to light illumination, and a time counter 108.

[0005] The time counter 108 starts counting time at the same time as the light source of the dToF system emits pulsed light 114. The light 114 reflects on the object 112 and returns to the photodetector as reflected light 116, and the photodetector outputs a signal. The time counter 108 stops counting time in response to the signal from the photodetector. When the dToF system includes a plurality of dToF sensors 102 arranged in an array, each dToF sensor 102 includes a photodetector, an amplifier, and a time counter. Therefore, since each dToF sensor 102 can measure the flight time of light, that is, the distance to the object, a two-dimensional depth image can be obtained by mapping the measured distance.

[0006] That is, if a portion of the light reflected on the object to be measured is further reflected on another object, the flight path of the reflected light is longer than the flight path of the light directly returning to the dToF sensor, and therefore, the time when such light returns to the photodetector is later than the light directly returning to the photodetector. Therefore, in the case where the reflected light is irradiated multiple times in one measurement, a multipath reflection problem may occur. In order to solve such a multipath reflection problem, the time counter of the dToF sensor stops counting the time in response to the output and amplification of the first signal from the photodetector. After stopping counting the time, the signal generated by the light irradiating the photodetector can be ignored. Therefore, the dToF sensor can detect the shortest flight time of light.

[0007] However, since the signal output from the dToF sensor may include false detection caused by dark current and ambient light, and changes in detection time caused by jitter, the dToF system usually obtains data from multiple frames and performs statistical processing on the data to obtain a depth image.

[0008] On the other hand, a conventional iToF system includes at least one iToF sensor 202. The iToF sensor 22 includes a photodetector (e.g., Fig.11 ), and measures the flight time of light based on the ratio between the signal stored in the taps 224, 226 and the signal output from the taps 224, 226. When the iToF system includes a plurality of iToF sensors, each iToF sensor can measure the flight time of light, that is, the distance to the object, and can obtain a two-dimensional depth image by mapping the measured distance.

[0009] For example, Fig.11 The conventional iToF sensor 202 shown in includes two taps 224, 226. The taps may generally include: a storage capacitor for storing signal charges output from a photodetector in response to light irradiation; and at least one gate for controlling the activation of the tap, the storage of signal light in the storage capacitor, and the output of the stored signal charges. When the light source of the iToF sensor 202 emits pulsed light 214, the tap 224 is activated synchronously with the light emission, and stores the signal charge generated by the light irradiating the photodetector. After the activation of the tap 224 is completed, the tap 226 is activated, and the signal charge generated by the light irradiating the photodetector is stored. After the measurement cycle is repeated at least once (preferably multiple times), the signal charges stored in the taps 224, 226 are output. In such an iToF sensor 202, when the light 214 reflected on the object 212 returns to the photodetector, the reflected light irradiates the photodetector both during the period of activating the tap 224 and during the period of activating the tap 226. For example, Fig.11It is shown that 80% of the reflected light 216 illuminates the photodetector during the period when the tap 224 is activated, and 20% of the reflected light 216 illuminates the photodetector during the period when the tap 226 is activated. If the object 212 moves away from the iToF sensor 202 and the flight time of the light becomes longer, the amount of reflected light 216 that illuminates the photodetector during the period when the tap 224 is activated decreases, and the amount of reflected light 216 that illuminates the photodetector increases during the period when the tap 226 is activated. Therefore, the flight time of light, that is, the distance to the object can be measured based on the ratio between the signal output from the tap 224 and the signal output from the tap 226.

[0010] By repeating the above measurement cycle multiple times and storing the charge in the tap, the results of the measurement cycle can be averaged. Therefore, the iToF system may not require the statistical process of multiple frames required by the dToF system. In addition, since the iToF system may not require a time counter for each iToF sensor, the iToF sensor can be miniaturized compared to the dToF sensor.

[0011] However, conventional iToF systems cannot avoid the effects of multipath reflections. Since light reflected from another object returns to the photodetector later than light directly reflected from the object, Fig.11 During the period shown when tap 226 is activated, the amount of reflected light illuminating the photodetector may increase. Thus, the ratio between the outputs of taps 224, 226 may deviate from the distance to the object, and the distance to the object may be determined to be greater than the actual distance.

[0012] As discussed above, since the time counter of the dToF system will hinder the miniaturization of the dToF sensor, the resolution of the depth image will be reduced. In addition, since there may be false detection caused by dark current and ambient light, and changes in detection time caused by jitter, the dToF system usually needs to use a statistical process of a large number of frames. In other words, since a large amount of data is output and processed, the system may require a large memory and a high-performance processing circuit.

[0013] On the other hand, the iToF system has the problem of multipath reflection. When multipath reflection occurs, the accuracy of distance measurement will be reduced. Summary of the invention

[0014] Therefore, high-resolution ToF sensors are required to solve the multipath reflection problem. The concept of dToF system detection is necessary to solve the multipath reflection problem, while the pixels must be miniaturized to obtain high resolution and the amount of output data must be reduced.

[0015] According to a first aspect of the present invention there is provided a method for measuring the distance from a photodetector to an object, the method performing a measurement cycle comprising:

[0016] The first step is performed once or multiple times, and the first step includes:

[0017] A first sub-step is for emitting pulsed light from at least one light source;

[0018] a second sub-step of sequentially activating each tap of a photodetector including a plurality of taps within a predetermined period of time to store a signal generated by the photodetector receiving light, wherein the plurality of taps are sequentially activated so that completion of activation of a last tap of the plurality of taps is offset from the emission of the pulsed light by a predetermined offset time greater than or equal to zero,

[0019] The second step is for outputting the stored signal from the tap and outputting a differential signal of the output signal,

[0020] In this process, the pulsed light reflects off the object and returns to the photodetector as reflected light.

[0021] wherein the measurement cycle is repeated as the offset time increases until the reflected light returns to the photodetector during an activation period of any one of the plurality of taps and the differential signal becomes a predetermined value greater than zero,

[0022] Therein, the distance from the photodetector to the object is measured based on an offset time of a measurement cycle in which reflected light returns to the photodetector during an activation period of any one of the plurality of taps.

[0023] In the first aspect of the present invention, the first step may be performed multiple times.

[0024] In the first aspect according to the present invention, the differential signal may be a digital signal expressed as Hi / Low, and when a signal output from a last activated tap among a plurality of taps is greater than signals output from other taps, the differential signal may be Hi.

[0025] In accordance with the first aspect of the present invention, when the differential signal exceeds a predetermined threshold, the distance from the photodetector to the object may be measured based on the offset time.

[0026] In the first aspect according to the invention, a differential signal may be stored in the resistor in each measurement cycle, and a signal may be output when the differential signal in one measurement cycle is greater than the differential signal stored in the resistor in a measurement cycle preceding the measurement cycle by a predetermined threshold.

[0027] In the first aspect of the present invention, the method may also include, when the differential signal output from each photodetector included in the pixel array is Hi or exceeds a predetermined threshold, forming a two-dimensional depth image by writing the offset time into an address of a frame memory, the address of the frame memory corresponds to the photodetector and does not include data about the offset time.

[0028] In the first aspect of the present invention, the method may further include forming a subframe of a two-dimensional depth image based on a differential signal output from each photodetector included in the pixel array, and data of only an address of a photodetector corresponding to an output signal in the subframe is updated.

[0029] According to a second aspect of the present invention, there is provided a time-of-flight (ToF) measurement system, comprising:

[0030] At least one ToF sensor;

[0031] Time counter;

[0032] at least one light source for emitting pulsed light,

[0033] Among them, ToF sensors include:

[0034] a photodetector including a plurality of taps, the photodetector being configured to output a signal generated by receiving light reflected on an object and returned as reflected light;

[0035] a comparator, the outputs from the plurality of taps being coupled to the comparator, the comparator being configured to output a differential signal of the outputs from the plurality of taps,

[0036] In which multiple taps are activated sequentially to store the signal generated by the photodetector receiving light,

[0037] wherein the plurality of taps are sequentially activated such that completion of activation of a last tap of the plurality of taps is offset from the start of emission of the pulsed light by an offset time greater than or equal to zero,

[0038] wherein the emission of light and the activation of the plurality of taps are performed with increasing offset times until the differential signal exceeds a predetermined value greater than zero,

[0039] Among them, the offset time is managed by the time counter,

[0040] Therein, the distance from the photodetector to the object is measured based on the drift time.

[0041] In the second aspect of the present invention, after the plurality of taps are activated a plurality of times, the plurality of taps may output the optical signals received by the photodetector and store the optical signals received by the photodetector.

[0042] In the second aspect according to the present invention, the comparator outputs a digital signal representing Hi / Low.

[0043] In a second aspect according to the present invention, when the differential signal exceeds a predetermined threshold, the distance from the photodetector to the object can be measured based on the offset time.

[0044] In the second aspect of the present invention, the ToF sensor may further include:

[0045] a resistor to store the differential signal for each measurement cycle;

[0046] A comparison circuit is used to compare the differential signal in one measurement cycle with the differential signal stored in the resistor in a measurement cycle before one measurement cycle, and to output a signal when the differential signal in one measurement cycle is greater than the differential signal stored in the resistor by a predetermined threshold.

[0047] In the second aspect according to the present invention, the photodetectors may be arranged in an array to form a pixel array, and the ToF measurement system may further include a logic circuit for forming a subframe of a two-dimensional depth image based on differential signals output from the photodetectors.

[0048] In accordance with a second aspect of the present invention, the photodetectors may be arranged in an array to form a pixel array, and the ToF measurement system may further include a logic circuit, which, if data about the offset time is not written to the address of the frame memory corresponding to the photodetector, is used to form a two-dimensional depth image by writing the offset time to the address when the differential signal output from each photodetector is Hi or exceeds a predetermined threshold.

[0049] According to a third aspect of the present invention, there is provided a stacked sensor chip, comprising a system according to any one of the first and second aspects, the stacked sensor chip comprising:

[0050] A pixel array chip including at least a photodetector;

[0051] An electronic circuit chip comprising at least one of a comparator, a time counter, a logic circuit, a resistor and a comparison circuit,

[0052] The pixel array chip and the electronic circuit chip are stacked and electrically coupled to each other.

[0053] In the third aspect of the present invention, the pixel array chip may further include an electronic circuit for implementing an indirect ToF measurement method.

[0054] Technical effects of the present invention:

[0055] According to an aspect of the present invention, a high-resolution ToF sensor that solves the multipath reflection problem is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1a A ToF system according to a first embodiment of the present invention is schematically shown.

[0057] Figure 1b FIG. 4 shows a signal output by a comparator of a ToF system according to a first embodiment of the present invention.

[0058] Figure 2a A first measurement cycle of a method for measuring a distance to an object by a ToF system according to a first embodiment of the present invention is shown.

[0059] Figure 2b A second measurement cycle of the method for measuring the distance to an object by the ToF system according to the first embodiment of the present invention is shown.

[0060] Figure 2c The third measurement cycle of the method for measuring the distance to an object by the ToF system according to the first embodiment of the present invention is shown.

[0061] Figure 2d The fourth measurement cycle of the method for measuring the distance to an object by the ToF system according to the first embodiment of the present invention is shown.

[0062] Figure 3a The signal output by the ToF system according to the first embodiment of the present invention is shown relative to the offset time t.

[0063] Figure 3b The signal output by the ToF system according to the first embodiment of the present invention is shown relative to the offset time t.

[0064] Figure 4 The method for generating a depth image by a ToF system according to the first embodiment of the present invention is schematically shown.

[0065] Figure 5 Shows Figure 4 The flowchart of the method for obtaining a subframe in a ToF system is shown.

[0066] Figure 6 FIG. 4 shows a schematic circuit diagram of a ToF sensor according to a second embodiment of the present invention.

[0067] Figure 7 A subframe in a pixel array including a ToF sensor according to a second embodiment of the present invention is schematically shown.

[0068] Figure 8A cross-sectional view of a stacked sensor chip according to a third embodiment of the present invention is shown.

[0069] Fig. 9 A perspective view of a stacked sensor chip according to a fourth embodiment of the present invention is shown.

[0070] Fig.10 A conventional dToF system is shown.

[0071] Fig.11 A conventional iToF system is shown. DETAILED DESCRIPTION

[0072] FIG1( a) schematically shows a time-of-flight measurement system (ToF system) 1 according to a first embodiment of the present invention. For simplicity, the ToF system 1 shown in FIG1 includes one time-of-flight measurement sensor (ToF sensor) 2. However, it should be noted that the ToF system 1 may include a plurality of ToF sensors 2. The ToF system 1 also includes at least one light source 4, a comparator 6, a time counter 8, and a logic circuit 10.

[0073] The ToF sensor 2 may include a photodetector 22 and a plurality of taps. Although the ToF sensor 2 of the example shown in FIG. 1( a) includes two taps 24 and 26, it should be understood that the ToF sensor 2 may include three or more taps. Each tap may include: a storage capacitor for storing signal charges generated by light irradiating the photodetector 22; an output port for outputting the stored signal charges; and a gate for controlling the storage of the signal charges in the storage capacitor and outputting the signal charges from the output port.

[0074] For example, in the example shown in FIG. 1( a), the at least one light source 4 may include a single light source 4. The light source 4 emits pulsed light at predetermined intervals. Although the light source 4 may emit light having any wavelength, infrared light may be preferred because infrared light is invisible to humans.

[0075] The outputs of the plurality of taps of the ToF sensor 2 are input to the comparator 6. In the example shown in FIG. 1(a), the signals output from the two taps 24 and 26 are input to the comparator 6 to generate a differential signal. The differential signal may be an analog signal obtained by replacing the signal output from the tap 24 with the signal output from the tap 26. Otherwise, when the signal output from the tap 26 is greater than the signal output from the tap 26 by a predetermined threshold or more, the differential signal may be a 1-bit digital signal represented by Hi, otherwise by Low, or represented by 1 / 0, as shown in FIG. 1(b).

[0076] The time counter 8 starts counting time in response to input of a start signal, stops counting time after a predetermined period of time has passed, and outputs the elapsed time. The operation of the time counter 8 in the ToF system 1 will be discussed below.

[0077] The logic circuit 10 controls the components of the ToF system 1 and measures the distance D to the object 12 based on the differential signal output from the comparator 6 and the elapsed time output from the time counter 8. If the comparator 6 outputs a digital signal, the signal output from the tap 26 can be detected more clearly, and the accuracy of measuring the time ts can be improved. In addition, it is preferred that the digital signal is output from the comparator 6 so that the logic circuit 10 processes the differential signal. In the following, the embodiment may be explained based on an example in which the comparator 6 outputs a digital signal. The method of measuring the distance D will be discussed below.

[0078] FIG. 2 schematically shows a method for a ToF system 1 to measure a distance D to an object 12 according to a first embodiment of the present invention.

[0079] Figure 2a The ToF system 1 is shown performing a first sub-step of emitting pulsed light 14 by the light source 4. The light 14 reflects off the object 12, returns to the ToF sensor 2 as reflected light 16 and illuminates the photodetector 22 after a time ts.

[0080] The ToF system 1 also performs sequential activation Figure 2a The second sub-step of the plurality of taps or taps 24, 26 in the example shown causes the last tap ( Figure 2a The timing of the completion of the activation of the tap 26 in the example shown matches the start of the emission of the pulsed light 14 in the first sub-step. In other words, the offset time t from the start of the emission of the pulsed light 14 by the light source 4 to the completion of the activation of the tap 26 is zero. Figure 2a In , the activation of taps 24, 26 is performed before the emission of light 14, for a time equal to the sum of the activation durations of the taps. Figure 2a In the embodiment, the second sub-step may be performed before the first sub-step.

[0081] In the second sub-step, the tap 24 is activated for a predetermined period of time and stores the signal charge generated due to the photodetector 22 receiving light during the period. After the activation of the tap 24, the tap 26 is subsequently activated and stores the signal charge generated due to the photodetector 22 receiving light during the period. The duration of the activation period of the tap 26 is preferably the same as the duration of the activation period of the tap 24. The first step including the first sub-step and the second sub-step may be performed once or multiple times. Preferably, performing the first step multiple times can average noise, such as changes in dark current, jitter of the timing circuit, and changes in the return time of the reflected light 16, and improve measurement accuracy.

[0082] exist Figure 2a In the example shown, the first step is completed before the reflected light 16 returns to the ToF sensor 2. Therefore, the signal charge generated by the light illuminating the photodetector 22 is not stored in the taps 24, 26, and only the charge generated by the dark current and the noise of the ambient light can be stored.

[0083] Thereafter, the ToF system 1 performs a second step, in which the stored signal is output from the taps 24 and 26 and input to the comparator 6 to output a differential signal. Since the signal charge generated by the light irradiating the photodetector 22 is not stored as described above, the differential signal is smaller, for example, represented by a Low value. Noise such as dark current and ambient light can be removed by subtracting the signal output from the tap 24 from the signal output from the tap 26.

[0084] A measurement cycle of the ToF system 1 comprises a first step and a second step.

[0085] After the measuring cycle is completed, another measuring cycle is executed again, such as Figure 2b As shown. Figure 2a Unlike the situation shown in , the activation of the taps 24, 26 is performed as follows, that is, the timing of the completion of the activation of the tap 26 is offset from the start of the emission of the light 14 by an offset time t. Therefore, Figure 2b It is shown that the second sub-step is performed after the first sub-step. The offset time t is managed by a time counter 8 . Figure 2b It is also shown that the first step is completed before the reflected light 16 returns to the ToF sensor 2. Therefore, since the signal charge generated by the light irradiating the photodetector 22 is not stored in the taps 24, 26, the differential signal output from the comparator 6 in the second step is small, for example, represented by a Low value.

[0086] After the measuring cycle is completed, another measuring cycle is executed again, such as Figure 2c As shown. Figure 2b Compared with the case of , the offset time t from the start of emission of the light 14 to the completion of activation of the tap 26 increases, so during the activation of the tap 26, the reflected light 16 returns to the ToF sensor 2. Therefore, the signal charge generated by the light irradiating the photodetector 22 is stored in the tap 26. For example, the differential signal output from the comparator 6 is represented by the value of Hi.

[0087] exist Figure 2c After the measurement cycle shown is completed, another measurement cycle is performed again, such as Figure 2dAs the offset time t further increases, the reflected light 16 returns to the ToF sensor 2 during the activation of the tap 24. Therefore, the signal charge generated by the light irradiating the photodetector 22 is stored in the tap 24. For example, the differential signal output from the comparator 6 is represented as a Low value.

[0088] Figure 3a and Figure 3b Graphs showing the differential signal output from the comparator 6, ie, the value obtained by subtracting the output of the tap 24 from the output of the tap 26 with respect to the offset time t, are shown. Figure 3a A differential signal obtained by subtracting a signal output from the tap 24 from a signal output from the tap 26 is shown as an analog signal. Figure 3b It shows a one-bit digital signal represented as Hi when the signal output from the tap 26 is greater than the signal output from the tap 24 by a predetermined threshold or more, otherwise represented as Low, or represented as 1 / 0. The solid line shown in FIG3(a) represents the differential signal when there is no multipath reflection, and the dotted line represents the differential signal when there is multipath reflection, wherein the light reflected on the object 12 is also reflected on another object before irradiating the photodetector 22. When multipath reflection occurs, the time when part of the reflected light returns to the photodetector is later than the reflected light directly returning to the photodetector. Therefore, the shape of the curve has a low peak with a long tail toward the direction of time passage. When the offset time t is the same as the return time ts of the reflected light 16, in other words, when the completion of the activation of the tap 26 is the same as the time of the return of the reflected light 16, the comparator 6 starts to output a positive signal, and the signal increases as the offset time t increases. When the offset time t is further increased and the reflected light 16 starts to irradiate the photodetector during the activation of the tap 24, the differential signal output from the comparator 6 starts to decrease. When the amount of reflected light 16 illuminating the photodetector during activation of tap 24 is greater than the amount of reflected light 16 illuminating the photodetector during activation of tap 26, the differential signal becomes negative. When the offset time t is further increased, the differential signal returns to zero. Figure 3b In the case of , even if the intensity of the reflected light 16 irradiating the photodetector 22 during the activation of the tap 26 decreases and the duration of the reflected light 16 becomes longer due to the multipath reflection, the intensity of the reflected light 16 irradiated during the activation of the tap 26 is still greater than the intensity of the reflected light 16 irradiated during the activation of the tap 24, so the comparator 6 continues to output the Low value. Therefore, even if multipath reflection occurs, the output signal relative to the offset time t does not change.

[0089] If a multipath problem occurs, in which light reflected on the object 12 is reflected on another object before illuminating the photodetector 22, the multipath reflected light 18 illuminates the photodetector 22 later than the reflected light 16 directly returning from the object 12 to the photodetector 22. Therefore, the pulse of the reflected light becomes longer than the pulse of the transmitted light 14. As the offset time t increases, this results in a lower peak and a longer tail of the differential signal of the comparator 6, as shown in FIG3. However, the offset time ts that improves the differential signal indicates the shortest time for the reflected light 16 to return to the photodetector 22, that is, the flight time of the reflected light 16 directly returning from the object (regardless of the multipath reflected light 18). Therefore, the distance D to the object 12 can be measured based on the offset time ts, which improves the differential signal without being affected by multipath reflections.

[0090] Since the distance D to the object 12 can be measured by the measuring time ts, it is not necessary to perform a measuring cycle after the offset time t exceeds the time ts, as Figure 2c and Figure 2d shown.

[0091] Figure 4 A method for the ToF system 1 according to the first embodiment to generate a depth image of a distance D two-dimensionally mapped to an object 12 is schematically shown.

[0092] Figure 4 It is shown that the ToF system 1 may include a pixel array 30 having a plurality of ToF sensors 2 arranged in an array. For example, the ToF system 1 may include a comparator 6 and a latch circuit 32 for each column of the pixel array 30. For example, the output of the ToF sensor 2 may be collected for each column input to the comparator 6. Only the ToF sensors 2 in a particular row of the pixel array 30 output signals from the tap to the comparator 6, and the differential signals may be sequentially output to the logic circuit 10 by the latch circuit 32 of each column. The time counter 8 manages the shift time t of the ToF sensor 2. The signals output from all rows may be stored in the frame memory 34 as subframes. The distance D from each ToF sensor 2 to the object 12 is measured by obtaining subframes while increasing the offset time t. By integrating the obtained subframes, a final two-dimensional depth image may be generated.

[0093] Figure 5 Shows Figure 4Flow chart of a method for obtaining a subframe in the ToF system 1 shown in FIG. 1 . In step 101, the logic circuit 10 performs a measurement loop for each ToF sensor 2 of the pixel array 30. In step 102, the ToF sensor 2 in each column and a row outputs a light receiving signal. In step 103, each differential signal output from the comparator 6 in each column is determined based on the light receiving signal from the ToF sensor 2 in each row. If the differential signal from the comparator 6 is Low, steps 104 to 106 are not performed, but step 107 is performed in a loop. If the differential signal is Hi, the loop enters step 104. In step 104, the subframe data of the address corresponding to the ToF sensor 2 is loaded from the frame memory 34 to the logic circuit 10. In step 105, it is determined whether data about the offset time t is written in the subframe data of the loaded address. If the data exists, the loop step 106 is not performed, but step 107 is performed. If the data is empty, in step 106, the data about the offset time t is written to the address. In step 107, if the acquisition of the differential signals of the ToF sensors 2 of all rows and all columns in the subframe is not completed, the loop returns to step 102 and outputs the light reception signal from the ToF sensor 2 of the next row or column. If the differential signals of the ToF sensor 2 in all rows and all columns are completely acquired and the subframe is completed, the loop returns to step 101 with the increase of the offset time t to perform the measurement cycle of the next subframe. For each ToF sensor 2 in each column in a certain row, the logical operations from steps 103 to 106 can be performed in parallel by providing a plurality of circuit groups. These steps are repeated a predetermined number of times or until the offset time t reaches a predetermined time, so as to generate a two-dimensional depth image, in which the offset time t of the return of the reflected light 16 is written to each address of the ToF sensor 2, that is, where each address of the ToF sensor 2 is mapped to the distance of the object.

[0094] but, Figure 4 and Figure 5 The method shown significantly increases the total amount of data of all subframes output from the pixel array 30 to the logic circuit 10 that controls writing data to the frame memory 34 in order to generate a two-dimensional depth image, because even if the data output from each ToF sensor 2 is one bit, the data from all ToF sensors 2 is transmitted to the logic circuit 10. In order to reduce the amount of data, an event-driven type system is known. The data output by each ToF sensor of the event-driven type system is monitored by each ToF sensor itself. Only when the output exceeds a threshold level, the pixel is excited and the address of the pixel is output. Since each subframe only includes the address of the pixel whose output exceeds the threshold level, this reduces the total amount of data output from the pixel array 30 to the logic circuit 10 that controls writing data to the frame memory 34.

[0095] Figure 6 FIG. 4 is a schematic circuit diagram of a ToF sensor 2 according to a second embodiment of the present invention, wherein the ToF sensor 2 implements an event-driven system. Figure 7 A schematic diagram showing a subframe including a pixel array 30 of the ToF sensor 2 according to the second embodiment when one pixel is excited is shown.

[0096] Figure 6 A ToF sensor 2 is shown comprising a photodetector 22 comprising a plurality of taps. Figure 6 The photodetector 22 shown includes, for example, two taps 24, 26, but the photodetector 22 may include three or more taps. The outputs from the taps 24, 26 are coupled to the comparator 6, for example.

[0097] Since the method for operating the ToF sensor 2 having such a configuration is similar to the method described with reference to FIG. 2 for the first embodiment, it will not be described in detail. Figure 6 As shown in FIG. 1 , the offset time t from the start of emission of the light 14 to the completion of activation of the second tap 26 increases, the ToF sensor 2 repeatedly performs emission of the light 14 and activation of the first tap 24 and the second tap 26. The signal output from the first tap 24 and the signal output from the second tap 26 are input to the comparator 6. If the offset time t is equal to the time ts from the emission of the light 14 to the return of the reflected light 16 reflected on the object 12, the reflected light 16 irradiates the photodetector 22 during the activation of the second tap 26, and the signal output from the comparator 6 becomes Hi. Therefore, the time ts from the emission of the light 14 to the return of the reflected light 16 to the photodetector 22, that is, the distance to the object 12, is measured.

[0098] Figure 6 The output of the comparator 6 shown is coupled to a resistor 36 and a comparison circuit 38. The resistor 36 stores the output of the comparator 6 in the previous subframe as a Hi or Low data value. The comparison circuit 38 compares the output of the comparator 6 and the data stored in the resistor 36. If the output of the comparator 6 is Low, the comparison circuit 38 does not output Hi, and the Low data value is stored in the resistor 36 before the next subframe. If the output of the comparator 6 is Hi when the resistor 36 is in the Low state, the comparison circuit 38 outputs Hi, and the resistor 36 stores Hi. The state in which the comparator circuit 38 outputs Hi is referred to as "the state in which the pixel including the ToF sensor 2 is excited". If the pixel is excited and the resistor 36 stores Hi, the comparison circuit 38 no longer outputs Hi, and the pixel is no longer excited until all subframes after the next subframe are completed and the entire pixel array is reset.

[0099] Figure 7A subframe is shown in which only pixel 22-1 in pixel array 30 is excited. When the row scan reaches the row including the excited pixel 22-1 in the subframe, the column corresponding to the excited pixel 22-1 is detected. Therefore, the data of the row and column of pixel 22-1 are output as addresses. When pixel 22-1 is excited, the offset time t (i.e., offset time t) of the subframe is written into the address corresponding to the address of pixel 22-1 in frame memory 34. Therefore, the distance from pixel 22-1 to object 12 is written into frame memory 34.

[0100] By repeating this operation a predetermined number of times, a two-dimensional depth image can be obtained.

[0101] Figure 8 A cross-sectional view of a stacked sensor chip 40 according to a third embodiment of the present invention is shown. The stacked sensor chip 40 includes a stacked structure including a pixel array chip 42 and an electronic circuit chip 44. The pixel array chip 42 may include photodetectors 22 arranged in an array, and a plurality of taps, such as taps 24, 26, coupled to each photodetector 22. The electronic circuit chip 44 may include: a control circuit for controlling the light source 4 and the taps 24, 26 included in the pixel array chip 42; a comparator 6 to which the outputs of the taps 24, 26 are coupled through a coupling portion 46; a time counter 8; a logic circuit 10; a resistor 36 and a comparison circuit 38, and may include other additional circuits. The photodetector 22 and the taps 24, 26 included in the pixel array chip 42 and the comparator 6 included in the electronic circuit chip 44 may constitute a ToF sensor 2.

[0102] The ToF system 1 according to an embodiment of the present invention may include various peripheral circuits, such as a control circuit, a compactor, a time counter, a logic circuit, a resistor, and a comparison circuit as described above. Therefore, if all components of the ToF system 1 are integrated in one chip, the miniaturization of each pixel will be hindered. However, by dividing the ToF system 1 into a pixel array chip 42 and an electronic circuit chip 44 and stacking them, each pixel and the entire ToF system 1 can be miniaturized. The operation of the ToF system 1 can also be accelerated by a parallel circuit. The delay of the operation can also be suppressed by reducing the distance between the photodetector 22 and the electronic circuit. In addition, since the pixel array chip 42 and the electronic circuit chip 44 are manufactured in a separate process and coupled to each other in the last step, the manufacturing process can be simplified. Since a wafer including a plurality of pixel array chips 42 and a wafer including a plurality of electronic circuit chips 44 are wafer bonded and divided into chips, the manufacturing process can be further simplified.

[0103] Fig. 9A perspective view of a stacked sensor chip 50 according to a fourth embodiment of the present invention is shown. The stacked sensor chip 50 according to the fourth embodiment includes a stacked structure in which a pixel array chip 52 including a plurality of ToF sensors 2 arranged in an array and an electronic circuit chip 54 are stacked together via a coupling portion 56.

[0104] Similar to the electronic circuit chip 44 of the stacked sensor chip 40 of the third embodiment, the electronic circuit chip 54 may include: Fig. 9 Circuit blocks 58 and 59 are shown. Circuit block 58 may include: a control circuit for controlling the taps of light source 4 and pixel array chip 52 to implement the operation of ToF sensor 2 described with reference to the first embodiment and the second embodiment; comparator 6; resistor 36; comparison circuit 38. Circuit block 60 may include time counter 8, logic circuit 10, and optionally other additional circuits.

[0105] On the other hand, the pixel array chip 52 may include an indirect TOF system logic circuit 62 for implementing a conventional indirect time-of-flight measurement method. Since the ToF sensor 2 included in the stacked sensor chip 50 according to the present embodiment includes a plurality of taps, the ToF sensor 2 may be used as a reference. Fig.11 The conventional indirect ToF sensor described above, etc. If necessary, such a stacked sensor chip 50 can select a suitable operation mode among the conventional direct ToF sensor mode, the conventional indirect ToF sensor mode and the ToF sensor mode according to an embodiment of the present invention.

[0106] Although the embodiments of the present invention have been described by way of example, it will be readily apparent to those skilled in the art that various modifications and variations can be made without departing from the spirit and scope of the invention.

[0107] name

[0108] 1: ToF system

[0109] 2: ToF sensor

[0110] 22: Photodetector

[0111] 22-1: Exciting pixels

[0112] 24, 26: Tap

[0113] 4: Light source

[0114] 6: Comparator

[0115] 8: Time counter

[0116] 10: Logic Circuit

[0117] 12: Objects

[0118] 14: Light

[0119] 16: Reflected Light

[0120] 18: Multipath reflection

[0121] 30: Pixel Array

[0122] 32: Latch circuit

[0123] 34: Frame memory

[0124] 36: Resistors

[0125] 38: Comparison circuit

[0126] 40: Stacked sensor chip

[0127] 42: Pixel array chip

[0128] 44: Electronic circuit chip

[0129] 46: Coupling part

[0130] 50: Stacked sensor chip

[0131] 52: Pixel array chip

[0132] 54: Electronic circuit chip

[0133] 56: Coupling part

[0134] 58, 60: Circuit block

[0135] 62: Logic circuit of indirect ToF system

[0136] 102: Traditional dToF Sensor

[0137] 108: Time counter

[0138] 112: Object

[0139] 114: Light

[0140] 116: Reflected Light

[0141] 202: Traditional iToF sensor

[0142] 212: Object

[0143] 214: Light

[0144] 216: Reflected Light

[0145] 224, 226: Tap

Claims

1. A method for measuring the distance from a photodetector to an object, characterized in that The method performs a measurement cycle, the measurement cycle comprising: The first step is performed once or multiple times, and the first step includes: A first sub-step is for emitting pulsed light from at least one light source; a second sub-step of sequentially activating each tap of the photodetector including a plurality of taps within a predetermined period of time to store a signal generated by the photodetector receiving light, wherein the plurality of taps are sequentially activated so that completion of activation of a last tap of the plurality of taps is offset from the emission of the pulsed light by a predetermined offset time greater than or equal to zero, a second step for outputting the stored signal from the tap and outputting a differential signal of the output signal, wherein the pulse light is reflected on the object and returns to the photodetector as reflected light, wherein the measurement cycle is repeated as the offset time increases until the reflected light returns to the photodetector during an activation period of any one of the plurality of taps and the differential signal becomes a predetermined value greater than zero, wherein the distance from the photodetector to the object is measured based on the offset time of the measurement cycle in which the reflected light returns to the photodetector during the activation period of any one of a plurality of taps.

2. The method according to claim 1, characterized in that The first step is performed multiple times.

3. The method according to claim 1, characterized in that The differential signal is a digital signal represented as Hi / Low. When the signal output from the last activated tap among the multiple taps is greater than the signals output from the other taps, the differential signal is Hi.

4. The method according to claim 1, characterized in that: When the differential signal exceeds a predetermined threshold, the distance from the photodetector to the object is measured based on the offset time.

5. The method according to claim 1, characterized in that In each measurement cycle, the differential signal is stored in the resistor, Wherein, when the differential signal in one measurement cycle is greater than the differential signal stored in the resistor in the measurement cycle before the one measurement cycle by a predetermined threshold, the signal is output.

6. The method according to any one of claims 1 to 5, characterized in that It also includes forming a two-dimensional depth image by writing the offset time into an address of a frame memory when the differential signal output from each photodetector included in the pixel array is Hi or exceeds a predetermined threshold, wherein the address of the frame memory corresponds to the photodetector and does not include data about the offset time.

7. The method according to claim 6, characterized in that Also included is a subframe that forms a two-dimensional depth image based on a differential signal output from each photodetector included in the pixel array, Among them, in the subframe, only the data corresponding to the address of the photodetector that outputs the signal is updated.

8. A time-of-flight (ToF) measurement system, characterized in that: include: At least one ToF sensor; Time counter; at least one light source for emitting pulsed light, Wherein, the ToF sensor comprises: a photodetector including a plurality of taps, the photodetector being configured to output a signal generated by receiving light reflected on an object and returned as reflected light; a comparator to which the outputs from the plurality of taps are coupled, the comparator being configured to output a differential signal of the outputs from the plurality of taps, wherein the plurality of taps are sequentially activated to store signals generated by the photodetector receiving light, wherein the plurality of taps are sequentially activated such that completion of activation of a last tap of the plurality of taps is offset from the start of emission of the pulsed light by an offset time greater than or equal to zero, wherein the emission of light and the activation of the plurality of taps are performed with increasing offset time until the differential signal exceeds a predetermined value greater than zero, Wherein, the offset time is managed by the time counter, Wherein, the distance from the photodetector to the object is measured based on the offset time.

9. The system according to claim 8, characterized in that After the plurality of taps are activated a plurality of times, the plurality of taps output the optical signals received by the photodetector and store the optical signals received by the photodetector.

10. The system according to claim 8, characterized in that The comparator outputs a digital signal representing Hi / Low.

11. The system according to claim 8, characterized in that When the differential signal exceeds a predetermined threshold, the distance from the photodetector to the object is measured based on the offset time.

12. The system according to claim 8, characterized in that The ToF sensor further comprises: a resistor to store the differential signal for each measurement cycle; A comparison circuit is used to compare the differential signal in one measurement cycle with the differential signal stored in the resistor during a measurement cycle before the one measurement cycle, and to output a signal when the differential signal in the one measurement cycle is greater than the differential signal stored in the resistor by a predetermined threshold.

13. The system according to claim 8, characterized in that The photodetectors are arranged in an array to form a pixel array, The ToF measurement system also includes logic circuitry for forming subframes of a two-dimensional depth image based on the differential signal output from the photodetector.

14. The system according to claim 12, characterized in that The photodetectors are arranged in an array to form a pixel array, When the differential signal output from each photodetector is Hi or exceeds a predetermined threshold, the ToF measurement system also includes a logic circuit for forming a two-dimensional depth image by writing the offset time into the address corresponding to the photodetector in the frame memory if the data about the offset time is not written into the address.

15. A stacked sensor chip, characterized in that: The system according to any one of claims 8 to 14, wherein the stacked sensor chip comprises: A pixel array chip, comprising at least the photodetector; an electronic circuit chip, comprising at least one of the comparator, the time counter, the logic circuit, the resistor and the comparison circuit, Wherein, the pixel array chip and the electronic circuit chip are stacked and electrically coupled to each other.

16. The stacked sensor chip according to claim 15, characterized in that: The pixel array chip also includes an electronic circuit for implementing an indirect ToF measurement method.

Images