Time-of-flight distance measurement sensing system and image sensor

Abstract

To provide a device capable of avoiding distortion of information obtained from each pixel due to distance difference between pixels in a pixel array of an image sensor employed in a time-of-flight (TOF) sensing system and a drive unit configured to drive the pixel array.SOLUTION: An image sensor employed in a time-of-flight (TOF) sensing system is provided, comprising: a pixel array 326A consisting of a plurality of pixels arranged in a plurality of rows and a plurality of columns, each pixel being configured to generate an amount of charge corresponding to incident light; and a first drive unit 322A configured to feed a control signal to a plurality of pixels via a plurality of columns. The first drive unit 322A is configured to feed the control signal via either odd columns or even columns.SELECTED DRAWING: Figure 9A

Description

[Technical Field]

[0001] The present invention relates to Time of Flight (TOF) sensing systems, and more particularly to an apparatus for improving the resolution or precision of a sensing system. [Background technology]

[0002] A Time of Flight (TOF) system can calculate the distance to an object or target by measuring the flight time of light or a signal. Time of flight refers to the time it takes for light or a signal to be emitted and reflected by the object or target. A TOF camera uses the TOF method to output an image containing information about the depth of an object based on the distance between the object and the camera. While conventional cameras output two-dimensional images that show the color, shape, etc. of an object or target, a TOF camera can output not only two-dimensional images of an object or target, but also three-dimensional images that show its depth. Summary of the Invention [Problem to be solved by the invention]

[0003] Embodiments of the present invention provide an apparatus that can prevent distortion of information obtained from each pixel due to a distance difference between a pixel provided in a pixel array in an image sensor used in a Time of Flight (TOF) sensing system and a driver that drives the pixel array.

[0004] The present invention also provides a device that enables a driving voltage output from a driver to be transmitted to a pixel array having a plurality of pixels arranged on a row and column basis in a time-of-flight (TOF) sensing system in an order that can compensate for differences in distance between the driver and each pixel.

[0005] In addition, the present invention provides a device in which one driver supplies a driving voltage to every two columns so as to compensate for operational differences between pixels due to delays, resistance, etc. that occur in the process of transmitting a driving voltage output from a driver as the size of an image sensor provided in a Time-of-Flight (TOF) sensing system increases.

[0006] The technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0007] The present invention provides a Time of Flight (TOF) based sensing system and image sensor.

[0008] An image sensor installed in a time-of-flight (TOF) sensing system according to an embodiment of the present invention includes a pixel array having a plurality of pixels arranged along a plurality of rows and a plurality of columns, each pixel generating an electric charge corresponding to received light, and a first driver supplying a control signal to the plurality of pixels via the plurality of columns, wherein the first driver can supply the control signal to one of odd and even columns of the plurality of columns.

[0009] The first driver may include a first structure for supplying the control signal to one end of the odd-numbered columns or a second structure for supplying the control signal to one end of the even-numbered columns, and the other end of the even-numbered columns may be connected to the other end of the odd-numbered columns.

[0010] The image sensor may further include a second driver that supplies the control signal to the other of the odd-numbered columns and the even-numbered columns of the plurality of columns.

[0011] The first driver and the second driver may be disposed on opposite sides of the pixel array.

[0012] Also, the distance between the pixel array and each of the first and second driving units may be the same.

[0013] The pixel also comprises at least one diode and at least one transfer gate, and the pixel comprises the same number of transfer gates as the number of the diodes, and the control signal can be supplied to the at least one transfer gate.

[0014] The pixel may also include a first receiver that senses the received light at a first phase and a second receiver that senses the received light at a second phase opposite to the first phase.

[0015] An image sensor installed in a time-of-flight (TOF) sensing system according to another embodiment of the present invention may include a pixel array having a plurality of pixels arranged along a plurality of rows and a plurality of columns, each pixel generating an electric charge corresponding to received light, a first driver supplying a first control signal to some pixels via some of the plurality of columns, and a second driver supplying a second control signal to the remaining pixels via columns of the plurality of columns to which the first control signal is not supplied.

[0016] Furthermore, if the first control signal is transmitted through odd-numbered columns among the plurality of columns, the second control signal may be transmitted through even-numbered columns among the plurality of columns, and if the first control signal is transmitted through the even-numbered columns, the second control signal may be transmitted through the odd-numbered columns.

[0017] Furthermore, the first control signal and the second control signal may be alternately supplied to the plurality of columns.

[0018] The first driver and the second driver may be disposed on opposite sides of the pixel array.

[0019] The pixel also comprises at least one diode and at least one transfer gate, and the pixel comprises the same number of transfer gates as the number of the diodes, and the control signal can be supplied to the at least one transfer gate.

[0020] The image sensor may further include a signal converter that converts the charge information output from the pixel array into a digital signal.

[0021] According to another embodiment of the present invention, a time-of-flight (TOF) sensing system includes a transmitter that outputs light having a predetermined phase, an image sensor that receives light reflected from an object, and a signal processing unit that determines a distance to the object based on the relationship between the light and the reflected light. The image sensor includes a pixel array that generates charge information corresponding to the reflected light and has a plurality of pixels arranged along a plurality of rows and a plurality of columns, a first driver that supplies control signals to the plurality of pixels via the plurality of columns, and a signal conversion unit that converts the charge information output from the pixel array into a digital signal and transmits the digital signal to the signal processing unit. The first driver may supply the control signal to one of odd and even columns of the plurality of columns.

[0022] The first driver may include a first structure for supplying the control signal to one end of the odd-numbered columns or a second structure for supplying the control signal to one end of the even-numbered columns, and the other end of the even-numbered columns may be connected to the other end of the odd-numbered columns.

[0023] The image sensor may further include a second driver that supplies the control signal to the other of the odd-numbered columns and the even-numbered columns of the plurality of columns.

[0024] The first driver and the second driver may be disposed on opposite sides of the pixel array.

[0025] Also, the distance between the pixel array and each of the first and second driving units may be the same.

[0026] The pixel also comprises at least one diode and at least one transfer gate, and the pixel comprises the same number of transfer gates as the number of the diodes, and the control signal can be supplied to the at least one transfer gate.

[0027] The pixel may also include a first receiver that senses the received light at a first phase and a second receiver that senses the received light at a second phase opposite to the first phase.

[0028] According to another embodiment of the present invention, an image sensor may include a pixel array including a plurality of pixels arranged in row and column lines, each pixel configured to generate a charge, the amount of charge corresponding to the amount of incident light sensed by the pixel. The image sensor includes a drive circuit connected to the column lines around the pixel array and configured to supply drive control signals having substantially the same period as the incident light to the pixels via each pair of odd and even columns. The sum of the amounts sensed by each pair of two pixels arranged on each row line may be substantially the same, and the pixel array may be configured to generate a sum of the charges generated by each pair of two pixels arranged on each row line.

[0029] The above aspects of the present invention are merely a portion of the preferred embodiments of the present invention, and various embodiments reflecting the technical features of the present invention can be derived and understood by those skilled in the art based on the detailed description of the present invention set forth below. [Effects of the Invention]

[0030] The effects of the device according to the present invention are as follows.

[0031] The Time of Flight (TOF) sensing system according to an embodiment of the present invention has the advantage of being able to measure the distance to an object more accurately and to generate a 3D image showing the depth of the object.

[0032] In addition, the present invention can compensate for the delay in the driving voltage caused by the position of the pixel that generates the charge based on the light reflected by the object and received, based on the order in which the driving voltage is transmitted, thereby providing an apparatus that can generate a 3D image more effectively and quickly than compensating for the delay in the driving voltage through a separate algorithm or signal processing method.

[0033] The effects that can be obtained in the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Brief explanation of the drawings]

[0034] [Figure 1] The operation of a Time of Flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 2] An example of the time difference and phase difference measured between a modulated signal and a reflected signal by a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 3] A first example of a time-of-flight (TOF) sensing system according to an embodiment of the present invention will be described. [Figure 4] A second example of a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 5]An example of a pixel included in an image sensor mounted on a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 6] The operation of an image sensor mounted on a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 7] A method for estimating distance and a method for determining the magnitude of a modulation signal in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 8] A delay occurring during operation of an image sensor mounted in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 9A] An example for improving delays that occur during operation of an image sensor mounted in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 9B] An example for improving delays that occur during operation of an image sensor mounted in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 9C] An example for improving delays that occur during operation of an image sensor mounted in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 9D] An example for improving delays that occur during operation of an image sensor mounted in a time-of-flight (TOF) sensing system according to an embodiment of the present invention will now be described. [Figure 10] The results of improved delay will be explained through the examples illustrated in FIGS. 9A to 9D. DETAILED DESCRIPTION OF THE INVENTION

[0035] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that in the following description, only the parts necessary for understanding the operation of the present invention will be described, and the description of the other parts will be omitted so as not to obscure the gist of the present invention.

[0036] Hereinafter, embodiments of the present invention will be described in more detail with reference to the drawings.

[0037] FIG. 1 illustrates the operation of a Time of Flight (TOF) sensing system according to an embodiment of the present invention.

[0038] 1, a time-of-flight (TOF) sensing system 100 can measure the distance to a target 20. The time-of-flight (TOF) sensing system 100 can include a transmitter 110 that outputs a modulated signal to measure the distance to the target 20, and a receiver 120 that receives a reflected signal of the modulated signal reflected by the target 20.

[0039] The time-of-flight (TOF) sensing system 100 can be one of the crucial elements in the development of automation in various industrial fields and consumer markets. Consumer devices can recognize their surrounding environment or recognize their position in the surrounding environment through the time-of-flight (TOF) sensing system 100. A camera including the time-of-flight (TOF) sensing system 100 can provide 3D vision technology by determining the color, shape, and distance of objects contained in a 3D environment.

[0040] The modulated signal output from the transmitter 110 in the time-of-flight (TOF) sensing system 100 emits a predetermined pattern into the surrounding environment, and when the reflected signal reflected from various objects in the surrounding environment is received by the receiver 120, the distance can be determined based on the correlation between the modulated signal and the reflected signal.

[0041] Ambient light can be a hindrance in the process of observing the relationship between the modulated signal and the reflected signal and determining distance based on that relationship. Particularly in bright sunlight conditions or when high resolution is required, additional efforts to remove interference (e.g., noise) from ambient light may be required to obtain distance data.

[0042] 2 illustrates an embodiment of the operation of a time-of-flight (TOF) sensing system 100 according to an embodiment of the present invention. Specifically, FIG. 2A illustrates an example in which the time-of-flight (TOF) sensing system measures a time difference or time delay between a modulated signal and a reflected signal, and FIG. 2B illustrates an example in which the time-of-flight (TOF) sensing system measures a phase difference or phase delay between a modulated signal and a reflected signal to estimate the time difference or time delay.

[0043] The time-of-flight (TOF) ranging system 100, which measures the time difference or time delay between a modulated signal and a reflected signal as shown in FIG. 2A, directly measures the round-trip time between the modulated signal output from the transmitter 110 as shown in FIG. 1, reflected by the target 20, and received by the receiver 120. Here, the modulated signal may have a predetermined pulse shape. Because the intensity of the pulse irradiance is greater than the intensity of the irradiance due to other background radiation, the method of measuring the time difference or time delay can be used in rural areas and is easy to measure long distances, making it suitable for use in autonomous vehicle LIDAR. However, since a very fast time measurement element (TDC) is required, this method is often expensive, and since it cannot measure time at many pixels, it generally has low resolution. As a result, it is primarily used for expensive specific purposes such as satellites, space exploration, and national defense.

[0044] The time-of-flight (TOF) ranging system 100, which measures the phase difference or phase delay between a modulated signal and a reflected signal as described in FIG. 2B, can estimate distance through simple signal processing to measure the phase delay. This system is easy to use for measuring relatively short distances of within a few meters and is primarily used indoors. Because a 3D image is obtained through an optical signal processing process within the sensor, it is easy to miniaturize, requires little computational effort, and can have a high frame rate. It can also be easily implemented with little installation space and relatively low cost. However, in the case of measuring the phase difference or phase delay, the measurable distance may be limited. For example, because the phase difference is used to measure distance, accurate measurement of an object at a distance greater than one period may be impossible (ambiguity problem).

[0045] FIG. 3 illustrates a first example of a time-of-flight (TOF) sensing system 100A according to an embodiment of the present invention. Specifically, FIG. 3 may include an example of a time-of-flight (TOF) sensing system using continuous wave modulation. If the time of flight can be directly measured through a modulated signal and a reflected signal, the distance between the time-of-flight (TOF) sensing system 100A and a target 20 can be easily determined using the speed of light (c). However, directly measuring the exact time of flight is difficult and prone to errors. Here, a time-of-flight (TOF) sensing system using continuous wave modulation can measure the phase difference between a transmitted signal and a received signal. In particular, the cross-correlation between a received signal and a transmitted signal, which can be recognized by a time-of-flight (TOF) sensing system, can enable phase estimation directly related to distance when the modulation frequency is known.

[0046] As shown in FIG. 3, a first example 100A of a time-of-flight (TOF) sensing system may include a transmitter 110, a receiver 120, a modulator 140, a demodulator 150, a signal processor 130, a signal converter 160, a first lens unit 170, and a second lens unit 180.

[0047] The signal processor 130 may receive a distance measurement request from another device or a user interface that is linked to the time-of-flight (TOF) sensing system, and then output data on the estimated distance. In response to the received request, the signal processor 130 may drive the modulator 140. For example, the signal processor 130 may output a modulation control signal MC to the modulator 140 to control the operation of the modulator 140.

[0048] In response to the output of the signal processing unit 130, the modulator 140 can generate a modulated signal to be output or emitted via the transmitter 110. According to an embodiment, the modulator 140 can include a modulation control unit 142 and an transmitter driving unit 144. The modulation control unit 142 can output a modulated periodic signal FC so that the transmitter 110 can generate a modulated signal that can be distinguished from ambient light described in FIG. 1 . According to an embodiment, the transmitter 110 can include a light-emitting element (e.g., a light-emitting diode), and the modulated periodic signal FC output by the modulator 140 can be used as a driving signal to drive the light-emitting element. According to yet another embodiment, the modulator 140 can control the transmitter 110 so that the modulated signal generated by the transmitter 110 has a predetermined frequency or a predetermined magnitude. To this end, the modulation control unit 142 can output a modulated periodic signal FC having a specific frequency or a specific phase to the transmitter 110.

[0049] The modulation controller 142 included in the modulator 140 receives the modulation control signal MC and generates a modulation period signal FC for generating various types of modulation signals. For example, the transmitter driver 144 can control the pulse of a light source or a light modulation element included in the transmitter 110 via the modulation period signal FC. Depending on the embodiment, the modulation controller 142 can control the light source or the light modulation element included in the transmitter 110 to output a modulation signal such as a triangular wave (e.g., a ramp waveform), a sine wave, or a square wave.

[0050] However, due to driving errors and nonlinearities of the light-emitting diode (LED) or laser diode (LD) included in the transmitter 110, it may be difficult for the transmitter 110 to generate an ideal waveform, such as a pulse, triangular wave, or sine wave, as specified. For example, a light-emitting diode (LED) operates above a threshold current, and nonlinearity and saturation of the output optical power relative to the input current may occur even within the driving range. Furthermore, a light-emitting diode (LED) may also have a nonlinear optical modulation gain within the driving range. In particular, when using a high voltage or high current, the nonlinearity or driving error of the LED may increase depending on the design of the driving circuit. Such driving errors may directly affect the distance information extraction results and cause errors in the distance sensed by the time-of-flight (TOF) sensing system 100A. Therefore, the modulation control unit 142 in the modulator 140 may include a complex additional algorithm and driving circuit to compensate for such errors.

[0051] The transmitter driver 144 may output a drive control signal DC for driving a light source and a light modulation element included in the transmitter 110. In response to the drive control signal DC, the transmitter 110 may output a modulation signal. According to an embodiment, the transmitter 110 may include a laser diode or the like that can output light in response to a control signal from the modulator 140. The modulation signal output from the transmitter 110 may have a frequency that belongs to the infrared or ultraviolet range, rather than the visible light range used to determine the color, shape, etc., of objects included in a 3D environment. For example, the transmitter 110 may include a light emitting diode (LED) or a laser diode (LD) for light of a specific wavelength (e.g., near-infrared light of 850 nm). Although FIG. 3 illustrates the transmitter driver 144 being included in the modulator 140, according to an embodiment, the transmitter driver 144 may be included in the transmitter 110 that includes a light emitting diode (LED) or a laser diode (LD). In addition, in another embodiment, the light emitting diode LED or the laser diode LD included in the transmitter 110 can be directly driven and controlled by the modulation controller 142 .

[0052] The modulated signal output from the transmitter 110 may be output to the outside of the sensing system 110A through the first lens unit 170. The first lens unit 170 may be implemented in various ways depending on the purpose of use and operating environment of the time-of-flight (TOF) sensing system. For example, the first lens unit 170 may output the modulated signal to a specific position or area, or may evenly diverge the modulated signal over a predetermined area.

[0053] The reflected signal may be transmitted to the receiver 120 via the second lens unit 180. Depending on the embodiment, the second lens unit 180 may collect the reflected signal and transmit it to the receiver 120. The second lens unit 180 may also include at least one filter for receiving only the reflected signal resulting from the modulated signal being reflected back by the target 20.

[0054] The receiver 120 may include a pixel array 126 in which a plurality of pixels are arranged in an array, and the pixel array 126 can receive a reflected signal and generate pixel information PI, a charge amount, or a signal corresponding to the reflected signal. The pixel array 126 arranged in the receiver 120 can be controlled by the demodulation unit 150 to output the plurality of pixel information PI, a charge amount, or a signal to the signal conversion unit 160.

[0055] The modulation period signal FC output from the modulation control unit 142 in the modulation unit 140 may include information about the modulation signal output through the first lens unit 170. The modulation period signal FC may be input to the demodulation unit 150. The demodulation unit 150 may output a drive control signal TC for controlling the receiver 120 in response to the modulation period signal FC. The demodulation unit 150 may determine different phases corresponding to the modulation period signal FC. According to an embodiment, the phase conversion unit 152 may output phase information PFC such as 0 degrees or 180 degrees. In another embodiment, the phase conversion unit 152 in the demodulation unit 150 may output phase information PFC such as 90 degrees, 180 degrees, 270 degrees, or 360 degrees to the driver 154. The driver 154 may transmit the drive control signal TC to the receiver 120 in response to the phase of the modulation signal determined by the phase conversion unit 152. Here, the receiver 120 may include a pixel array 126. The reflected signal collected by the receiver 120 by the phase converter 152 and driver 154 in the demodulator 150 will be described in detail below with reference to Figures 6 and 7. Thus, since the demodulator 250 already knows the characteristics of the modulated signal through the modulation period signal FC, it can drive the receiver 120 through a phase shift to measure, collect, or determine the reflected signal.

[0056] The receiver 120 may be controlled by a drive control signal TC output from the demodulator 150, which may generate the drive control signal TC in response to a modulated period signal FC for controlling the transmitter 110. A phase converter in the demodulator 150 outputs a modulated phase signal PFC in response to the modulated period signal FC, and the driver 154 may generate the drive control signal TC in response to the modulated phase signal PFC. Here, the modulated phase signal PFC may include a plurality of signals having a predetermined phase difference. The driver 154 may output the drive control signal TC in response to the modulated phase signal PFC for driving a plurality of pixels in a pixel array 126 included in the receiver 120.

[0057] The pixel information PI, charge information, charge amount, or signal output from the receiver 120 may be converted into data via the signal converter 160. For example, the pixel information PI, charge information, charge amount, or signal transmitted via the receiver 120 and the demodulator 150 may be analog data AD, and the output converted via the signal converter 160 may be digital data DD. The data DD converted by the signal converter 160 is transmitted to the signal processor 130.

[0058] The signal processor 130 can estimate the distance between the time-of-flight (TOF) sensing system and the target 20 through a calculation process based on the data transmitted from the signal converter 160. The operation of the signal processor 130 will be described later with reference to FIGS. 6 and 7. In addition, the signal processor 130 can calculate the depth of an object contained in a predetermined area based on the estimated distance information within the predetermined area. For example, if the distance between the time-of-flight (TOF) sensing system and the target 20 at a first position within the predetermined area is 3 m and the distance between the time-of-flight (TOF) sensing system and the target 20 at a second position is 3.5 m, the depth between the first and second positions can be estimated to be 50 cm.

[0059] Fig. 4 illustrates a second example 100B of a time-of-flight (TOF) sensing system according to an embodiment of the present invention. Compared to the first example 100A of the time-of-flight (TOF) sensing system illustrated in Fig. 3, the second example 100B of the time-of-flight (TOF) sensing system illustrated in Fig. 4 can have a structure that simplifies the modulation signal and can be used in small devices.

[0060] 4, the second example 100B of the time-of-flight (TOF) sensing system may include a transmitter 210, a receiver 220, a transmitter driver 244, a demodulator 250, a signal processor 230, a signal converter 260, a first lens unit 270, and a second lens unit 280. Depending on the embodiment, the demodulator 250 may include a phase converter 252 and a driver 254.

[0061] The components of the second example 100B of the time-of-flight (TOF) sensing system described in Fig. 4 may be similar to the components of the first example 100A of the time-of-flight (TOF) sensing system described in Fig. 3. For example, the transmitters 210, 110, receivers 220, 120, demodulators 150, 250, first lens units 270, 170, and second lens units 280, 180 described in Fig. 3 and Fig. 4 are not significantly different in function and role, and therefore detailed description in Fig. 4 will be omitted.

[0062] The signal processor 230 may receive a distance measurement request from another device or a user interface linked to the time-of-flight (TOF) sensing system and output data on the estimated distance. After receiving a request to acquire distance information, the signal processor 230 may transmit a modulation period signal FC having a modulation frequency to the transmitter driver 244. The transmitter driver 244 may output a modulated signal in response to the modulation period signal FC.

[0063] In addition, the modulated period signal FC having the modulation frequency output from the signal processing unit 230 may be input to the demodulator 250. The demodulator 250 may output a drive control signal TC for controlling the receiver 220 in response to the modulated period signal FC. The demodulator 250 may determine different phases corresponding to the modulated period signal FC. For example, the phase converter 252 in the demodulator 250 may output phase information PFC, such as 90 degrees, 180 degrees, 270 degrees, or 360 degrees, to the phase signal generator 240. The driver 254 may transmit a drive control signal TC to the receiver 220 in response to the phase of the modulated signal determined by the phase converter 252. Here, the receiver 220 may include a pixel array 126. The reflected signal collected by the receiver 220 through the phase converter 252 and the driver 254 in the demodulator 250 will be described with reference to FIGS. 6 and 7.

[0064] Since the demodulator 250 already knows the characteristics of the modulated signal through the modulated periodic signal FC, it can measure, collect, or determine the reflected signal by driving the receiver 220 through a phase shift.

[0065] The receiver 220 may measure, collect, or determine a reflected signal in response to the drive control signal TC transmitted from the demodulator 250 and output pixel information PI. The pixel information PI is transmitted to the signal converter 260, which may output digital data DD to the signal processor 230. The signal processor 230 may calculate or obtain distance information based on the digital data DD, and the operation of the signal processor 230 will be described with reference to FIGS. 6 and 7.

[0066] Depending on the embodiment, the time-of-flight (TOF) sensing system 100A, 100B may include various types of circuits and algorithms for generating modulated signals output via the transmitters 110, 210 and drive signals for driving the pixel array 126 included in the receivers 120, 220 in response to the modulated signals.

[0067] The time-of-flight (TOF) sensing systems 100A and 100B described in FIGS. 3 and 4 can be distinguished by the modulation controller 142. According to an embodiment, the modulation controller 142 described in FIG. 3 can operate to diversify the modulated signal output from the transmitter 110 and reduce errors in the modulated signal. In the case of the time-of-flight (TOF) sensing system 100A including the modulation controller 142, the modulation signal can be diversified to facilitate distance measurement in a wider variety of environments. For example, if it is determined that there is significant error in distance measurement in a particular environment, the signal processor 130 can control the modulation controller 142 via the modulation control signal MC to output a modulation period signal FC for generating a modulated signal having a different form or frequency.

[0068] FIG. 5 illustrates an example of pixels provided in an image sensor mounted on a time-of-flight (TOF) sensing system according to an embodiment of the present invention.

[0069] 5, the image sensor 520 may include a pixel array 126 including a plurality of pixels 128 and a driver 122 for driving the pixel array 126. Within the pixel array 126, the plurality of pixels 128 may be arranged along a plurality of rows and a plurality of columns.

[0070] Depending on the embodiment, each of the plurality of pixels 128 may be implemented differently. A first example pixel 128A may include one light receiving element D and one transmission gate TG, and a second example pixel 128B may include two light receiving elements D1, D2 and two transmission gates TG1, TG2. The capacitances C, C1, and C2 of each pixel shown in FIG. 5 may refer to the capacitances of the photodiodes D, D1, and D2 of each pixel 128. As the pixel size (e.g., well capacitance) increases, the capacitances C, C1, and C2 may increase.

[0071] In the second example of pixel 128B, two nodes (or two taps) may exist in one pixel where photocharges corresponding to input light, such as the reflected signals described in Figures 1 to 4, are collected, and two transmission gates TG1 and TG2 can be controlled with signals having opposite phases. Furthermore, controlling one pixel with two opposite-phase signals has the advantage of increasing the light-receiving area of ​​the pixel that receives input light, which may be advantageous in increasing the resolution of the image sensor, rather than controlling two adjacent pixels with two opposite-phase signals.

[0072] FIG. 6 illustrates the operation of an image sensor mounted on a time-of-flight (TOF) sensing system 100 according to an embodiment of the present invention.

[0073] 6, each of a plurality of pixels 128 included in an image sensor 520 mounted on a time-of-flight (TOF) sensing system 100 may include a pair of receivers 312A, 328B. Depending on the embodiment, as shown in FIGS. 5 and 6, one pixel 128 may include a pair of receivers 312A, 328B, and two adjacent pixels 128 may be driven as a pair.

[0074] As described in Figures 1 to 4, a modulated signal output from a time-of-flight (TOF) sensing system 100 can be reflected by a target 20 and then received as a reflected signal.

[0075] The pair of receivers 328A, 328B can include a first phase receiver 328A and a second phase receiver 328B that can be activated in opposite phases to each other to output a photocharge corresponding to a reflected signal (e.g., light).

[0076] For example, the modulated signal may be configured in a pattern in which the transmitters 110 and 210 are turned on and output during half of one period, and are turned off and not output during the remaining half period (dotted line). Thus, the modulated signal has an active region for one half period, and an inactive region for the other half period. After flying to the target 20, the modulated signal may be reflected by the target 20 and received by the receivers 120 and 220. The reflected signal and the modulated signal received by the receivers 120 and 220 may have a phase difference corresponding to the time of flight.

[0077] According to an embodiment, the first phase receiver 328A may be activated for a time corresponding to a half period in which the transmitters 110 and 210 are turned on and a modulated signal is output, thereby outputting a photocharge corresponding to the reflected signal, and the second phase receiver 328B may be activated for a time corresponding to another half period in which the transmitters 110 and 210 are turned off and a modulated signal is not output, thereby outputting a photocharge corresponding to the reflected signal. Here, the activation times of the first phase receiver 328A and the second phase receiver 328B may be the same.

[0078] When the reflected signal and the modulated signal have a phase difference as shown, part of the reflected signal can be received through the first phase receiver 328A and the rest through the second phase receiver 328B. By comparing the amount of the reflected signal received through the first phase receiver 328A with the amount of the reflected signal received through the second phase receiver 328B, the distance traveled by the reflected signal can be estimated.

[0079] For example, although not shown, assume that the distance traveled by the modulated signal is 0 m. In this case, the amount of the reflected signal received via the first phase receiver 328A may be 100% and the amount of the reflected signal received via the second phase receiver 328B may be 0%.

[0080] The distance traveled by the modulated signal may correspond to the frequency (period) of the modulated signal and the speed of light. For example, if the frequency of the modulated signal is 1 Hz, the period of the modulated signal is 1 second. If the amount of the reflected signal received through the first phase receiver 328A is 0% and the amount of the reflected signal received through the second phase receiver 328B is 100%, it can be estimated that the modulated signal has traveled for 0.5 seconds. In this case, assuming that the time it takes the modulated signal to travel to the target and the time it takes the reflected signal to return are the same, the distance between the time-of-flight (TOF) sensing system and the target can be determined by multiplying half the time-of-flight (0.25 seconds) by the speed of light.

[0081] 7 illustrates a method for estimating distance and a method for determining the magnitude of a modulated signal in a time-of-flight (TOF) sensing system according to an embodiment of the present invention. As illustrated in FIGS. 6 and 7, the method for estimating distance and the method for determining the magnitude of a modulated signal may be an example of an internal operation of the signal processors 130 and 230 described in FIGS. 3 and 4.

[0082] As shown in Fig. 7, it is assumed that a modulated signal and a reflected signal have a phase difference φ. As described in Figs. 1 to 4, a modulated signal output from a time-of-flight (TOF) sensing system 100 can be received as a reflected signal after being reflected by a target 20.

[0083] For example, the modulated signal described in FIG. 7 can be understood as a signal generated by the transmitter 110, 210 described in FIGS. 3 and 4 and output through the first lens unit 270, and the reflected signal described in FIG. 7 can be understood as a signal transmitted to the receiver 120, 220 through the second lens unit 180, 280 described in FIGS. 3 and 4.

[0084] The time-of-flight (TOF) sensing system 100 can shift the phase of a modulated signal. For example, it can generate phase shifts of 0°, 90°, 180°, and 270°. As described in FIG. 6, 0° and 180°, and 90° and 270° are opposite phases and can be received via a pair of receivers 120 and 220.

[0085] For example, the phase transition signals C1, C2, C3, and C4 for distance measurement described in FIG. 7 can be understood as the drive control signals TC output from the demodulation units 150 and 250 described in FIGS. 3 and 4 and transmitted to the receivers 120 and 220.

[0086] The amount of charge generated in each pixel 128 may be Q1, Q2, Q3, or Q4, corresponding to the amount of reflected signal received corresponding to each phase transition (0°, 180°, 90°, or 270°). For example, first pixel information Q1 may be output from a pixel 128 controlled by a first phase transition signal C1, and second pixel information Q2 may be output from another pixel 128 controlled by a second phase transition signal C2. In an embodiment in which the pixels 128 have a two-tab configuration, the first pixel information Q1 and the second pixel information Q2 may be output from the same pixel 128 but through different transmission gates that can be individually controlled by drive signals having opposite phases. In another embodiment in which each pixel 128 has one photodiode and one transmission gate, the first pixel information Q1 and the second pixel information Q2 may be individually output from two adjacent pixels 128 that can be individually controlled by being driven by signals having opposite phases. For example, the amounts of charge Q1, Q2, Q3, and Q4 corresponding to the reflected signals illustrated in FIG. 7 can be understood as pixel information PI output from the receivers 120 and 220 illustrated in FIGS.

[0087] Based on this, the phase difference φ that can be determined by the signal processing units 130 and 230 described with reference to FIGS. 3 and 4 can be determined by the following equation.

number

[0088] Once the phase difference φ is determined, the speed of light c and the frequency f of the modulating signal can be calculated. mod The distance can be estimated based on this.

number

[0089] In addition, the amplitude of the modulation signal can be estimated based on the amounts of charge Q1, Q2, Q3, and Q4 corresponding to the reflected signal, which is pixel information PI.

number

[0090] Through the above-described method, the time-of-flight (TOF) sensing system 100 can determine the frequency or magnitude of the modulated signal and estimate the distance to the target 20. In addition, the frequency or magnitude of the modulated signal can be changed depending on the operating environment of the time-of-flight (TOF) sensing system 100.

[0091] Meanwhile, the operation of the above-mentioned Time-of-Flight (TOF) sensing system 100 can reduce errors in the estimated distance and increase the resolution by shifting the phase of the modulated signal and receiving the reflected signal accurately for each phase.

[0092] As illustrated in FIG. 1 , the environment in which the time-of-flight (TOF) sensing system 100 operates may contain not only natural light or ambient light but also modulated signals and reflected signals. For example, the modulated signal used in the time-of-flight (TOF) sensing system 100 may have a frequency band in the ultraviolet or infrared range. The modulated signal may also have the same frequency as the natural light or optical signal. Even if the second lens unit 180, 280 includes a filter, not only the reflected signal but also natural light or ambient light having the corresponding frequency band in the ultraviolet or infrared range may be input to the time-of-flight (TOF) sensing system 100. In this case, the natural light or ambient light may act as noise or interference. Therefore, when the time-of-flight (TOF) sensing system 100 is used outdoors during the day, the receivers 120, 220 may receive too much incident light due to the presence of natural light or ambient light. If too much incident light is input to the receiver 120, 220, the time-of-flight (TOF) sensing system 100 may not have the operating margin to calculate or estimate the distance to the target or the target's depth information.

[0093] FIG. 8 illustrates a delay that occurs during operation of the image sensor 520 mounted on the time-of-flight (TOF) sensing system 100 according to an embodiment of the present invention.

[0094] 8, the image sensor 520 may include a pixel array 126 including a plurality of pixels and a driver 122 for driving the pixel array 126. As shown in FIG. 5, the pixel array 126 may include a plurality of pixels 128 arranged along a plurality of rows and a plurality of columns.

[0095] The driver 122 can provide a control signal (e.g., a preset voltage) to turn on the transmission gate (TG, TG1, TG2, see FIG. 5) of each pixel 128. The driver 122 can determine the control signal provided to each pixel in response to the clock signal Clk or the modulation signal.

[0096] As shown in FIG. 8, assume that a control signal output from the driver 122 is supplied to the transmission gate of each pixel along one column of the pixel array 126. Ideally, the voltage supplied to the transmission gate of each pixel would be transmitted to all pixels at the same time; however, a signal delay (RC delay) may occur along the line transmitting the voltage. Therefore, a time difference may occur between the voltage TXnear supplied to the transmission gate of the pixel closest to the driver 122 and the voltage TXfar supplied to the transmission gate of the pixel farthest from the driver 122. In this case, a difference occurs in the drive time of each pixel receiving a reflected signal, i.e., the time to output charge corresponding to the reflected signal.

[0097] Even if the reflected signals received by the pixel closest to the driver 122 and the pixel farthest from the driver 122 in the pixel array 126 are the same (same distance), if there is a difference in the drive time of each pixel due to signal delay depending on the pixel's position (i.e., distance from the driver 122), a problem may arise in which the estimated distance corresponding to the reflected signal received by each pixel changes, as described in Figures 6 and 7.

[0098] In particular, when the time-of-flight (TOF) sensing system 100 attempts to more accurately estimate the distance to the target 20 through various stages (between various periods of the modulation signal and the reflected signal), the difference in the amount of reflected signal received by the pixel closest to the driver 122 and the pixel furthest from the driver 122 in the pixel array 126 can gradually increase. That is, the difference between the closest pixel and the furthest pixel can gradually increase as various stages are integrated (signal integration) due to the difference in the signal sensed at each stage. The following equation is for n stages: When the pixel is moved forward, the sum of the nearest pixels

number

number

number

[0099] Therefore, in a time-of-flight (TOF) sensing system, a method of measuring for various periods in order to estimate a more accurate distance may result in an inaccurate distance estimate.

[0100] To solve this problem, the line size can be increased to reduce the resistance in order to reduce the delay in the line that transmits the control signal (e.g., a preset voltage) supplied to the transmission gate of each pixel. While this method has the effect of reducing the resistance and reducing the delay, it can be burdensome to increase the line size in a fine process.

[0101] 9A to 9D illustrate an example of improving delays that occur during operation of an image sensor installed in a time-of-flight (TOF) sensing system 100 according to an embodiment of the present invention. In FIGS. 9A to 9D, the order in which voltages are supplied to pixels is changed to compensate for delays that occur when control signals transmitted by a driver are transmitted through lines. According to an embodiment, pixel arrays 326A, 326B, 326C, and 326D include a plurality of pixels, each of which receives a control signal from a driver as illustrated in FIGS. 9A to 9D. Each of pixel arrays 326A, 326B, 326C, and 326D illustrated in FIGS. 9A to 9D may include pairs of receivers (e.g., first phase receiver 328A and second phase receiver 328B; see FIG. 6) that can output charge information corresponding to received light in opposite phases of a control signal.

[0102] As shown in FIG. 9A , the driver 322A supplies a control signal through one side of the pixel array 326A, and odd and even columns are connected to each other on the other side of the pixel array 326A to form pairs. For each pair, the control signal output from the driver 322A is supplied to the odd columns on one side of the pixel array 326A and then transmitted to the even columns on the other side of the pixel array 326B via a line connecting the odd columns to the even columns. In this case, the control signal may be supplied earliest to the transmission gate of the pixel closest to the driver 322A in the odd columns, and latest to the transmission gate of the pixel closest to the driver 322A in the even columns. The control signal may then be supplied at an average time to the transmission gate of the pixel farthest from the driver 322A in the odd columns and the transmission gate of the pixel farthest from the driver 322A in the even columns. In this case, by integrating adjacent pixels in odd and even columns in the same row, delays that occur when pixels arranged in different rows in the pixel array 326A are closer or farther from the driver 322A can be compensated for, and the problem of errors occurring in the process of calculating the distance between the time-of-flight (TOF) sensing system 100 and the target 20 due to the pixel drive time changing depending on the pixel's position from the driver 322A can be solved.

[0103] 9B, the driver 322B supplies a control signal through one side of the pixel array 326B, and even and odd columns are connected to the other side of the pixel array 326B to form pairs. For each pair, the control signal output from the driver 322B is supplied to one end of the even column of the pixel array 326B and then transmitted to the odd column via a line connected from the other end of the even column to the odd column. In this case, the control signal may be supplied earliest to the transmission gate of the pixel closest to the driver 322B in the even column, and latest to the transmission gate of the pixel closest to the driver 322B in the odd column. The control signal may then be supplied at an average time to the transmission gate of the pixel farthest from the driver 322B in the even column and the transmission gate of the pixel farthest from the driver 322B in the odd column. In this case, by integrating adjacent pixels in odd and even columns in the same row, delays that occur when pixels arranged in different rows in the pixel array 326B are closer or farther from the driver 322B can be compensated for, and the problem of errors occurring in the process of calculating the distance between the time-of-flight (TOF) sensing system 100 and the target 20 due to the pixel drive time changing depending on the pixel's position from the driver 322B can be solved.

[0104] As shown in FIG. 9C , the image sensor 520 may include a first driver 322C and a second driver 324C disposed on opposite sides (opposite sides) of a pixel array 326C. Specifically, the first driver 322C and the second driver 324C may supply control signals to transmission gates of each pixel in the pixel array 326C in opposite directions. The first driver 322C may supply control signals to odd-numbered columns of the pixel array 326C, and the second driver 324C may supply control signals to even-numbered columns of the pixel array 326C. Even if delays occur in the odd and even columns, by integrating adjacent pixels in the odd and even columns in the same row, delays caused by the proximity or distance of pixels disposed in different rows of the pixel array 326C from the first driver 322C and the second driver 324C can be compensated for. In this case, the distance between each of the first driver 322C and the second driver 324C and the pixel array 326C may be the same. According to one embodiment, in order to prevent distortion due to delay differences between pixels, pixel information output individually from pixels arranged in odd-numbered rows and other pixels arranged in even-numbered rows may be combined or added together to compensate for the delay differences. This may reduce errors that occur in the process of calculating the distance between the time-of-flight (TOF) sensing system 100 and the target 20.

[0105] As shown in FIG. 9D, the image sensor 520 may include a first driver 322D and a second driver 324D disposed on opposite sides (opposite sides) of a pixel array 326D. Specifically, the first driver 322D and the second driver 324D may supply voltages in opposite directions to the transmission gates of each pixel in the pixel array 326D. The first driver 322D may supply control signals to the even columns of the pixel array 326D, and the second driver 324D may supply control signals to the odd columns of the pixel array 326D. Even if delays occur in the even and odd columns, by integrating adjacent pixels in the even and odd columns in the same row, delays caused by the proximity or distance of pixels disposed in different rows of the pixel array 326D from the first driver 322D and the second driver 324D can be compensated for. In this case, the distance between each of the first driver 322D and the second driver 324D and the pixel array 326C may be the same. According to one embodiment, in order to prevent distortion due to delay differences between pixels, pixel information output individually from pixels arranged in odd-numbered rows and other pixels arranged in even-numbered rows may be combined or added together to compensate for the delay differences. This may reduce errors that occur in the process of calculating the distance between the time-of-flight (TOF) sensing system 100 and the target 20.

[0106] As shown in FIG. 9D, the first drive circuit 322D and the second drive circuit 324D are arranged on opposite sides of the pixel array 326D. Similar to the embodiment shown in FIG. 9C, the first drive circuit 322D and the second drive circuit 324D can individually supply first and second drive control signals to the pixel array 326D and control the transmission gates of multiple pixels in the pixel array 326D. The first and second drive control signals are sequentially transmitted in opposite directions to pixels arranged in each row of the pixel array 326D. The first drive circuit 322D can supply first drive control signals to even columns of the pixel array 326D, and the second drive circuit 324D can supply second drive control signals to odd columns of the pixel array 326D. The sum of the distances between two adjacent pixels in the even and odd columns of each row of the pixel array 326D and the pixels of the first and second drive circuits 322D and 324D can be the same. Even if delays occur separately in the even and odd columns, the distance or delay due to different rows changing from one another within the pixel array 326D can be eliminated, and distortion due to delay differences between pixels can be reduced.

[0107] FIG. 10 illustrates the results of improved delay through the example illustrated in FIGS. 9A to 9D.

[0108] 10, even if a time difference occurs between the voltage TXnear supplied to the transmission gate of the pixel located closest to the driver and the voltage TXfar supplied to the transmission gate of the pixel located farthest from the driver, the order in which the voltage is supplied can be changed to compensate for the line delay of the voltage transmitted by the driver, as shown in FIGS. 9A to 9D. In this case, the same effect as essentially integrating the signals of two pixels corresponding to the voltage TXnear supplied to the transmission gate of the pixel located closest to the driver and the voltage TXfar supplied to the transmission gate of the pixel located farthest from the driver can be achieved.

[0109] In FIG. 10, it has been explained that the drive times corresponding to the voltage TXnear supplied to the transmission gate of the pixel located closest to the driver in the first period and the voltage TXfar supplied to the transmission gate of the pixel located farthest from the driver in the second period are integrated, but the drive times corresponding to the voltage TXnear supplied to the transmission gate of the pixel located closest to the driver and the voltage TXfar supplied to the transmission gate of the pixel located farthest from the driver can also be integrated for each period.

[0110] Therefore, unlike the description in FIG. 8, in the embodiments presented in FIGS. 9A to 9D, even if the distance is calculated over a plurality of steps (or various cycles), the difference in driving time between the pixel closest to the driver and the pixel farthest from the driver does not gradually increase. Even if the measurement is performed over various cycles, i.e., even if the signals of each step are integrated, the driving time does not differ depending on the pixel position within the image sensor 520 and can be the same. Therefore, even if the amount of reflected signals sensed by the image sensor 520 increases, the problem of the amount of reflected signals output individually from the farthest and nearest pixels changing can be reduced. The following equation is the sum of the nearest pixels when proceeding through n steps:

number

number

number

[0111] In the pixel array of the image sensor 520, the transmission of drive control signals from the driver may be delayed depending on the position of the pixel, but the above-described method can compensate for the delay that occurs. The above-described method is not intended to prevent delays that occur when the operating intervals change depending on the different distances between each pixel and the driver, but can provide a method and apparatus that can compensate for the delay that occurs through signal integration even if a delay occurs. This method can compensate for delays that may occur depending on whether the pixel is located close or far from the driver in the pixel array, and can solve problems that may occur when the pixel drive time changes depending on the position.

[0112] While the detailed description of the present invention has been given with reference to specific embodiments, it goes without saying that various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the following claims as well as equivalents to the claims.

Claims

1. In an image sensor mounted on a time-of-flight (TOF) sensing system, a pixel array that generates an electric charge corresponding to received light and includes a plurality of pixels arranged along a plurality of rows and a plurality of columns; a first driver that supplies control signals to the pixels via the columns; Equipped with the first driver supplies the control signal to one of odd-numbered columns and even-numbered columns of the plurality of columns; a first structure in which the first driver supplies the control signal to one end of the odd-numbered column; or the first driving unit includes a second structure for supplying the control signal to one end of the even-numbered columns, and the other ends of the pixels in the even-numbered columns and the other ends of the pixels in the odd-numbered columns are connected to each other in pairs.

2. The image sensor of claim 1 , further comprising a second driver that supplies the control signal to the other of the odd-numbered columns and the even-numbered columns of the plurality of columns that is not connected to the first driver.

3. The image sensor according to claim 2 , wherein the first driving unit and the second driving unit are arranged on opposite sides of the pixel array.

4. 3. The image sensor according to claim 2, wherein the distance between the pixel array and each of the first and second driving sections is the same.

5. the pixel comprises at least one diode and at least one transmission gate, the pixel comprising the same number of transmission gates as the number of the diodes; 2. The image sensor of claim 1, wherein the control signal is supplied to the at least one transmission gate.

6. The pixel is a first receiver for sensing the received light at a first phase; a second receiver for sensing the received light at a second phase opposite to the first phase; The image sensor of claim 1 , comprising:

7. a transmitter that outputs light having a preset phase; an image sensor that receives reflected light from an object; a signal processor that determines a distance to the object based on a relationship between the light and the reflected light; Equipped with The image sensor includes: a pixel array that generates charge information corresponding to the reflected light and includes a plurality of pixels arranged along a plurality of rows and a plurality of columns; a first driver that supplies control signals to the pixels via the columns; a signal conversion unit that converts the charge information output from the pixel array into a digital signal and transmits the digital signal to the signal processing unit; Equipped with the first driver supplies the control signal to one of odd-numbered columns and even-numbered columns of the plurality of columns; a first structure in which the first driver supplies the control signal to one end of the odd-numbered column; or A time-of-flight (TOF) sensing system in which the first driving unit includes a second structure that supplies the control signal to one end of the even columns, and the other ends of the pixels in the even columns and the other ends of the pixels in the odd columns are connected to each other in pairs.

8. The image sensor includes: The time-of-flight (TOF) sensing system of claim 7, further comprising a second driver that supplies the control signal to one of the odd and even columns of the plurality of columns to which the first driver is not connected.

9. The time-of-flight (TOF) sensing system according to claim 8 , wherein the first driving unit and the second driving unit are disposed on opposite sides of the pixel array.

10. The time-of-flight (TOF) sensing system of claim 8 , wherein the distances between the first driving unit and the pixel array are the same as the distances between the second driving unit and the pixel array.

11. the pixel comprises at least one diode and at least one transmission gate, the pixel comprising the same number of transmission gates as the number of the diodes; 8. The time-of-flight (TOF) sensing system of claim 7, wherein the control signal is provided to the at least one transmission gate.

12. The pixel is a first receiver for sensing the received light at a first phase; a second receiver for sensing the received light at a second phase opposite to the first phase; 8. The time-of-flight (TOF) sensing system of claim 7, comprising:

Images