Image sensing device, its operation method, and image sensing method

Abstract

To provide an image sensing device that adjusts an initial phase offset between a signal output from an optical transmitter and a signal for controlling an optical receiver.SOLUTION: An image sensing device comprises: an optical transmitter 110 which irradiates a subject with a modulation signal MS: an optical receiver 120 which generates a pixel signal PIX corresponding to a reflection signal RS from the subject according to a phase control signal; an image processor 140 which calculates a depth phase based upon the pixel signal, compensates the depth phase according to a phase offset signal OFF_PH, and outputs the result as depth information TOF_OUT; a drive control circuit 130 which generates a phase control signal corresponding to the phase of the modulation signal, and sets an initial phase difference between the phase control signal and the modulation signal according to the phase offset signal; and an offset calculation part 150 which sets a plurality of reference phases, selects a reference phase closest to the depth phase among the set reference phases, and calculates a phase offset signal corresponding to a phase difference between the selected reference phase and the depth phase.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to semiconductor design technology, and more specifically, to an image sensing device that acquires depth information by a TOF method.

Background Art

[0002] The time-of-flight (TOF) method refers to a method of calculating a distance by measuring the time-of-flight of light or a signal, that is, the time it takes for light or a signal to be emitted and reflected by a subject. A TOF sensor can acquire an image containing information regarding the depth of an object based on the distance between the object and the camera using the TOF method. If a conventional image sensor outputs a two-dimensional image representing the color, shape, etc. of an object, a TOF sensor can output a three-dimensional image representing not only the color and shape of the object but also the depth.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Embodiments of the present invention provide an image sensing device and an operation method thereof that can adjust an initial phase offset between a modulation signal output from a light transmitter and a phase control signal for controlling a light receiver based on depth information acquired by a TOF method.

Means for Solving the Problems

[0004] According to an aspect of the present invention, an image sensing apparatus includes: an optical transmitter 110 that irradiates a subject with a modulation signal MS; an optical receiver 120 that generates a pixel signal PIX corresponding to a reflection signal RS reflected from the subject according to a phase control signal PCTRL; an image processor 140 that calculates a depth phase TOP_PH based on the pixel signal and compensates the depth phase according to a phase offset signal OFF_PH to output depth information TOF_OUT; a drive control circuit 130 that generates the phase control signal corresponding to the phase of the modulation signal and sets an initial phase difference between the phase control signal and the modulation signal according to the phase offset signal; and an offset calculation circuit 150 that sets a plurality of reference phases, selects a reference phase closest to the depth phase among the set reference phases, and calculates the phase offset signal corresponding to a phase difference between the selected reference phase and the depth phase.

[0005] According to an aspect of the present invention, an image sensing method includes: a step of irradiating a subject with a modulation signal by an optical receiver; a step of generating, by an optical transmitter, a pixel signal corresponding to a reflection signal reflected from the subject according to a phase control signal; a step of generating a depth phase based on the pixel signal; a step of setting a plurality of reference phases, selecting a reference phase closest to the depth phase among the set reference phases, and calculating a phase offset signal corresponding to a phase difference between the selected reference phase and the depth phase; a step of setting an initial phase difference between the phase control signal and the modulation signal according to the phase offset signal; and a step of compensating the depth phase according to the phase offset signal and outputting the compensated depth phase as depth information.

[0006] According to one aspect of the present invention, a method for driving an image sensing device includes generating a source signal and irradiating a target therewith, receiving a reflected signal which is the source signal reflected from the target, generating a control signal PCTRL having a lagging phase delayed by a specific amount Td specified according to an offset signal OFF_PH with respect to the source signal, controlling a unit pixel via the control signal and generating a pixel signal PIX based on the reflected signal, calculating a depth phase according to the pixel signal, compensating the depth phase according to the offset signal to generate depth information TOF_OUT, selecting a reference phase having the smallest offset from among a plurality of reference phases from the depth information, and updating an offset signal corresponding to the smallest offset.

Advantages of the Invention

[0007] In an image sensing device that obtains depth information using a TOF method, according to an embodiment of the present invention, after setting an initial phase offset between a modulation signal and a phase control signal based on the depth information, the obtained depth information is finally compensated using the set initial phase offset, thereby minimizing a depth error and having an effect of being able to obtain accurate depth information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings in order to explain in detail to such an extent that a person having ordinary knowledge in the technical field to which the present invention pertains can easily implement the technical idea of the present invention.

[0010] Throughout the specification, when a part is described as "connected" to another part, this includes not only the case where it is "directly connected", but also the case where it is "electrically connected" with other circuits interposed therebetween. Also, when a part is described as "including" or "comprising" a certain component, this means that, unless otherwise stated to the contrary, it can further include or comprise other components, rather than excluding other components. Further, throughout the description in the specification, even if some components are described in the singular form, it will be understood that the present invention is not limited thereto and the components can be composed of a plurality of elements.

[0011] FIG. 1 is a diagram for explaining a TOF-type image sensing device 10.

[0012] As shown in FIG. 1, the TOF-type image sensing device 10 can measure the distance to a subject 20. The image sensing device 10 can be one of the extremely important elements in the automation development in various industrial fields and consumer markets. Devices used by consumers can recognize the surrounding environment or the position of the device in the surrounding environment via the image sensing device 10. A camera including the image sensing device 10 can provide three-dimensional vision technology by judging the color, form, and distance of things etc. included in a three-dimensional environment.

[0013] The image sensing device 10 can include a transmitter 12 that outputs a modulation signal and a receiver 14 that receives a reflected signal that is reflected by the subject 20 and returns after the modulation signal is reflected by the subject 20, in order to measure the distance to the subject 20. If the modulation signal output by the transmitter 12 is diffused into the surrounding environment in a preset pattern and the reflected signal reflected from various things and objects in the surrounding environment is received via the receiver 14, the distance can be determined through the correlation relationship (e.g., phase difference) between the modulation signal and the reflected signal. On the other hand, when determining the distance through the correlation relationship between the modulation signal and the reflected signal, an operation for removing noise caused by ambient light or the like can be performed.

[0014] FIG. 2 is a block diagram for explaining a TOF type image sensing device 100 according to the first embodiment of the present invention.

[0015] As shown in FIG. 2, the image sensing device 100 can generate depth information TOF_OUT representing the distance to the subject 200 using the TOF method. For example, the image sensing device 100 can generate depth information TOF_OUT by detecting the phase difference between a modulation signal MS output to the subject 200 and a reflected signal RS reflected from the subject 200.

[0016] The image sensing device 100 can include an optical transmitter 110, an optical receiver 120, a drive control circuit 130, an image processor 140, and an offset calculation circuit 150.

[0017] The optical transmitter 110 can output a modulation signal MS to the subject 200 according to the modulation control signal MC which is a source signal. For example, the modulation signal MS can be a periodic signal (e.g., a clock signal) that toggles periodically. According to an embodiment, the optical transmitter 110 can sequentially irradiate the subject 200 with modulation signals MS having different phases from each other. For example, the modulation signal MS can have at least two phases among a phase of 0 degrees (i.e., 360 degrees), a phase of 90 degrees, a phase of 180 degrees, and a phase of 270 degrees. Hereinafter, 0 degrees can have the same meaning as 360 degrees. The optical transmitter 110 can be composed of a Light Emitting Diode (LED) or a Laser Diode (LD). The modulation signal MS output from the optical transmitter 110 can have a frequency belonging to the infrared or ultraviolet region, which is not in the region of visible light used to determine the color, form, etc. of things included in the three-dimensional environment. For example, the optical transmitter 110 can irradiate light of a specific wavelength (e.g., near-infrared light of 850 nm).

[0018] The optical receiver 120 can receive the reflected signal RS reflected from the subject 200. The reflected signal RS can be input to the optical receiver 120 after a delay time corresponding to the time of flight from the modulation signal MS. The optical receiver 120 can include a pixel array 121 that can generate pixel signals PIXs corresponding to the reflected signal RS. The optical receiver 120 can remove noise due to ambient light, etc. from the reflected signal RS and provide the reflected signal RS with the noise removed to the pixel array 121. In particular, in the embodiment of the present invention, the pixel array 121 can constitute a TOF sensor. In FIG. 2, the optical receiver 120 is shown as including the pixel array 121, but the pixel array 121 can be arranged as a configuration separate from the optical receiver 120.

[0019] The pixel array 121 can measure, collect, or determine the reflection signal RS according to the low control signal RCTRL and the phase control signal PCTRL to generate the pixel signal PIXs. The pixel array 121 can include at least one unit pixel (not shown, PX in FIG. 3) for measuring the distance to the subject 200. The unit pixel PX can include at least two pixels (not shown, TAPA and TAPB in FIG. 3). For example, the unit pixel PX can be selected according to the low control signal RCTRL and generate the first and second pixel signals PIX1 and PIX2 corresponding to the reflection signal RS according to the phase control signal PCTRL. The detailed configuration of the pixel array 121 will be described with reference to FIG. 3.

[0020] The drive control circuit 130 can generate the modulation control signal MC, the low control signal RCTRL, and the phase control signal PCTRL in response to a request RQ for distance measurement from an external device or a user interface that operates in conjunction with the image sensing device 100. The drive control circuit 130 can generate a modulation control signal MC for adjusting the period of the modulation signal MS in response to the request RQ, and generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS. In particular, the drive control circuit 130 can set the initial phase difference (i.e., the initial phase offset) between the phase control signal PCTRL and the modulation signal MS according to the phase offset signal OFF_PH which is an offset signal. For example, the drive control circuit 130 can adjust (e.g., delay) the initial phase of the phase control signal PCTRL according to the phase offset signal OFF_PH to set the initial phase offset between the phase control signal PCTRL and the modulation signal MS. Thereby, the phase difference between the reflection signal RS and the phase control signal PCTRL can be set to a target value. At this time, the target value can be set to a value at which the depth error is minimized with respect to the depth phase TOF_PH, and desirably, it can be selected to be one of 0 degrees, 90 degrees, 180 degrees, or 270 degrees.

[0021] According to an embodiment, the drive control circuit 130 can include a modulation controller 132, a phase conversion circuit 134, a transmission drive unit 136, and a sensor drive unit 138.

[0022] The modulation controller 132 can generate a first drive control signal DRV_M in response to an external request RQ. The modulation controller 132 can generate the first drive control signal DRV_M such that the optical transmitter 110 can generate a modulation signal MS having a predetermined period that can be distinguished from ambient light. The modulation controller 132 can provide the first drive control signal DRV_M to the transmission drive unit 136 and the phase conversion circuit 134.

[0023] The transmission drive unit 136 can generate a modulation control signal MC in response to the first drive control signal DRV_M. According to an embodiment, the transmission drive unit 136 can be included in the optical transmitter 110, or the optical transmitter 110 can be directly driven and controlled by the modulation controller 132.

[0024] The phase conversion circuit 134 can generate a second drive control signal DRV_DD so that the sensor drive unit 138 operates in conjunction with the operation timing of the transmission drive unit 136 that operates in response to the first drive control signal DRV_M. Since the phase conversion circuit 134 already knows the characteristics of the modulation control signal MC via the first drive control signal DRV_M, it can determine different phases (e.g., 0 degrees, 90 degrees, 180 degrees, or 270 degrees) corresponding to the modulation signal MS. For example, the phase conversion circuit 134 can change (e.g., shift) the phase of the modulation signal MS based on the first drive control signal DRV_M to generate phase information PFC having different phases (e.g., 0 degrees, 90 degrees, 180 degrees, or 270 degrees) of the modulation signal MS, and can output the generated phase information PFC included in the second drive control signal DRV_DD. In particular, in the first embodiment, the phase conversion circuit 134 can change the phase information PFC such that the initial phase of the phase control signal PCTRL is delayed by a delay time Td in response to the phase offset signal OFF_PH. Thereby, the phase difference between the reflection signal RS and the phase control signal PCTRL can be set to a target value.

[0025] The sensor driving unit 138 can generate a row control signal RCTRL for controlling the pixel array 121 row by row according to the second driving control signal DRV_DD. For example, the sensor driving unit 138 can generate a first row control signal for controlling the unit pixel PX arranged in the first row of the pixel array 121, and can generate an nth row control signal for controlling the unit pixel PX arranged in the nth row of the pixel array 121 ("n" is a natural number greater than 2). Also, the sensor driving unit 138 can generate a phase control signal PCTRL for controlling the operation timing of the unit pixel of the pixel array 121 according to the second driving control signal DRV_DD. According to an embodiment, the phase control signal PCTRL can be a control signal having a lagging phase with respect to the modulation signal MS by an amount of delay specified by the phase offset signal OFF_PH. The sensor driving unit 138 can generate a phase control signal PCTRL whose phase is delayed with respect to the phase of the modulation signal MS by a delay time Td corresponding to the phase offset signal OFF_PH based on the phase information PFC included in the second driving control signal DRV_DD. The phase control signal PCTRL can correspond to the pixels TAPA and TAPB in the unit pixel PX respectively, and can include a plurality of phase transition signals (not shown, MIXA and MIXB in FIG. 3) having different phases from each other. For example, the sensor driving unit 138 can sequentially generate the first and second phase transition signals MIXA and MIXB corresponding to 0 degrees and 180 degrees of the modulation signal MS and the first and second phase transition signals MIXA and MIXB corresponding to 90 degrees and 270 degrees of the modulation signal MS based on the phase information PFC to drive the optical receiver 120. At this time, since the phase information PFC is changed according to the phase offset signal OFF_PH, the initial phases of the first and second phase transition signals MIXA and MIXB can be delayed from the modulation signal MS. According to an embodiment, the sensor driving unit 138 can be included in the optical receiver 120, or the optical receiver 120 can also be directly driven and controlled by the sensor driving unit 138.

[0026] The image processor 140 can generate depth information TOF_OUT representing the distance to the subject 200 based on the pixel signal PIXs. For example, the image processor 140 can calculate a depth phase TOF_PH based on the first and second pixel signals PIX1 and PIX2 provided from a unit pixel PX, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH. The image processor 140 can provide the calculated depth phase TOF_PH to the offset calculation circuit 150. In particular, when a phase offset signal OFF_PH is input, the image processor 140 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH, and calculate depth information TOF_OUT corresponding to the compensated depth phase TOF_PH.

[0027] According to an embodiment, the image processor 140 can include a signal conversion circuit 142 and a signal processing circuit 144.

[0028] The signal conversion circuit 142 can convert the first and second pixel signals PIX1 and PIX2 output from the optical receiver 120 into data DD in digital form. For example, the first and second pixel signals PIX1 and PIX2 output from the optical receiver 120 can be data in analog form, and the output converted through the signal conversion circuit 142 can be data DD in digital form.

[0029] The signal processing circuit 144 can perform a predetermined calculation process on the data DD transmitted from the signal conversion circuit 142 to calculate the depth phase TOF_PH, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH calculated based on the speed of light c and the frequency (period) of the modulation signal MS. The generated depth information TOF_OUT can be output as information representing the distance (i.e., depth) between the image sensing device 100 and the subject 200. Also, the signal processing circuit 144 can reflect the phase offset signal OFF_PH in the depth phase TOF_PH to compensate the depth phase TOF_PH by the amount of the phase changed by the phase conversion circuit 134, and calculate depth information TOF_OUT corresponding to the compensated depth phase.

[0030] The offset calculation circuit 150 can calculate a phase offset signal OFF_PH based on the depth phase TOF_PH fed back from the image processor 140. The offset calculation circuit 150 can set a plurality of reference phases, select the reference phase closest to the depth phase TOF_PH among the set reference phases, and generate a phase offset signal OFF_PH corresponding to the phase difference between the selected reference phase and the depth phase TOF_PH. For reference, the plurality of reference phases are phases at which the depth error is minimized with respect to the depth phase TOF_PH, and can be set to phases of 0 degrees, 90 degrees, 180 degrees, or 270 degrees.

[0031] FIG. 3 is a diagram for explaining the detailed configuration of the pixel array 121 of FIG. 2 according to the first embodiment of the present invention.

[0032] As shown in FIG. 3, the pixel array 121 can be arranged in an array form with a plurality of unit pixels PX. For example, one unit pixel PX can include first and second pixels TAPA and TAPB. The first and second pixels TAPA and TAPB are selected according to the low control signal RCTRL, and can generate first and second pixel signals PIX1 and PIX2 corresponding to the reflected signal RS according to the phase control signal PCTRL. For reference, in FIG. 3, the low control signal RCTRL can correspond to a reset signal RX, a transmission signal TX, and a selection signal SX, and the phase control signal PCTRL can correspond to first and second phase transition signals MIXA and MIXB. That is, the first and second pixels TAPA and TAPB can operate at different phases of the phase control signal PCTRL.

[0033] The first pixel TAPA can generate a first pixel signal PIX1 according to a reset signal RX, a transmission signal TX, a selection signal SX, and a first phase transition signal MIXA. For example, the first pixel TAPA can include a first sensing circuit P1, a first reset circuit RT1, a first transmission circuit TT1, a first charge storage circuit C1, a first driving circuit DT1, and a first selection circuit ST1. The first sensing circuit P1 is connected between a first node N1 and a low voltage terminal and can generate a charge according to the first phase transition signal MIXA. The first sensing circuit P1 can include a photodiode. The first reset circuit RT1 is connected between a high voltage terminal and the first node N1 and can reset the first sensing circuit P1 and the first charge storage node C1 according to the reset signal RX. The first transmission circuit TT1 is connected between the first node N1 and a first floating diffusion node FD1 and can reset the first charge storage circuit C1 according to the transmission signal TX and transmit the charge generated from the first sensing circuit P1 to the first charge storage circuit C1. The first charge storage circuit C1 can be connected between the first floating diffusion node FD1 and the low voltage terminal. The first charge storage circuit C1 can be a parasitic capacitor. The first driving circuit DT1 is connected between the high voltage terminal and the first selection circuit ST1 and can drive a first column line COL1 with a high voltage supplied through the high voltage terminal by a voltage applied to the first floating diffusion node FD1. The first selection circuit ST1 is connected between the first driving circuit DT1 and the first column line COL1 and can selectively connect the first driving circuit DT1 and the first column line COL1 according to the selection signal SX.

[0034] With the above structure, the first pixel TAPA can operate as follows. During the reset period determined according to the reset signal RX, the first reset circuit RT1 can reset the first charge storage circuit C1 and the first sensing circuit P1. During the transfer period determined by the transfer signal TX and the first phase transition signal MIXA, the first sensing circuit P1 can generate charges according to the first phase transition signal MIXA, and the first charge storage circuit C1 can store the charges. During the read period determined according to the selection signal SX, the first driving circuit DT1 and the first selection circuit ST1 can read the first pixel signal PIX1 corresponding to the charges to the image processor 140 via the first column line COL1.

[0035] The second pixel TAPB can generate the second pixel signal PIX2 according to the reset signal RX, the transfer signal TX, the selection signal SX, and the second phase transition signal MIXB. Since the detailed structure and operation of the second pixel TAPB are substantially the same as those of the first pixel TAPA, detailed description is omitted.

[0036] Figure 4 is a timing diagram for explaining the operation of a general TOF image sensing device. Figure 4 shows a case where the phase offset signal OFF_PH provided from the offset calculation circuit 150 in Figure 3 is not reflected.

[0037] As shown in Figure 4, if the light transmitter 110 irradiates the subject 200 with a modulation signal MS that toggles at a predetermined period, the modulation signal MS can be received as a reflection signal RS after being reflected by the subject 200. At this time, the modulation signal MS and the reflection signal RS can have a phase difference φ corresponding to the flight time.

[0038] The phase conversion circuit 134 can generate phase information PFC of 0 degrees, 90 degrees, 180 degrees, and 270 degrees of the modulation signal MS by shifting the phase of the modulation signal MS at 90-degree intervals. The sensor driving unit 138 can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS based on the phase information PFC. The sensor driving unit 138 can sequentially generate the first and second phase transition signals MIXA and MIXB corresponding to 0 degrees and 180 degrees of the modulation signal MS, and the first and second phase transition signals MIXA and MIXB corresponding to 90 degrees and 270 degrees of the modulation signal MS to drive the optical receiver 120.

[0039] The amounts (the amount of charge generated by the amount of light) of the reflected signals RS received corresponding to the sequentially input first and second phase transition signals MIXA and MIXB can be Q1, Q2, Q3, Q4. At this time, the amounts Q1 and Q3 of the reflected signal RS can correspond to the first pixel signal PIX1 in FIG. 3, and the amounts Q2 and Q4 of the reflected signal RS can correspond to the second pixel signal PIX2 in FIG. 3. The image processor 140 can calculate the depth phase TOF_PH based on the first and second pixel signals PIX1 and PIX2 provided from the unit pixel PX, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH.

[0040] On the other hand, in an image sensing device that acquires depth information by the TOF method, the depth error may increase as the depth increases, that is, as the distance becomes farther. At this time, in an ideal case, the depth error should increase linearly as the distance from the subject 200 increases. However, due to reasons such as the performance of the optical receiver and transmitter, pixel mismatch within the unit pixel, and mismatch of elements (for example, transistors) within the pixel, non-linear depth errors occur for each image sensing device (A, B). For example, as shown in the graph of FIG. 5A, if the target depth is 1000 mm, the depth error of device A becomes larger than the depth error of device B. As a result, it is difficult to compensate for the depth error collectively, and there is a difficulty that the depth error must be calculated and reflected each time depth information is acquired for each device.

[0041] Further, when generating the depth information TOF_OUT according to the pixel signals PIX1 and PIX2 corresponding to the amounts Q1, Q2, Q3, and Q4 of the reflected signal RS, from the viewpoint of the signal-to-noise ratio (SNR), when the intervals occupied by the amounts Q1, Q2, Q3, and Q4 of the reflected signal RS in the activation interval of the phase control signal PCTRL are 50:50 or 100:0, the performance of the image processor can be optimized most. That is, as shown in the graph of FIG. 5B, at the points where the phase difference between the reflected signal RS and the phase control signal PCTRL corresponds to 0 degrees, 90 degrees, 180 degrees, and 270 degrees, the signal-to-noise ratio (SNR) becomes 50:50 or 100:0, and the depth error can be minimized.

[0042] Therefore, in the embodiments of the present invention, the offset calculation circuit 150 is used to delay the initial phase of the phase control signal PCTRL, and the depth error can be minimized by setting the phase difference between the reflected signal RS and the phase control signal PCTRL to 0 degrees, 90 degrees, 180 degrees, or 270 degrees. That is, in the embodiments of the present invention, the offset calculation circuit 150 sets 0 degrees, 90 degrees, 180 degrees, and 270 degrees as the first to fourth reference phases, selects the reference phase closest to the depth phase TOF_PH among the set first to fourth reference phases, and can generate a phase offset signal OFF_PH corresponding to the phase difference between the selected reference phase and the depth phase TOF_PH. For example, as shown in FIG. 6A, assume that the depth phase TOF_PH fed back from the default state image processor 140 that does not reflect the phase offset signal OFF_PH is 108 degrees. In this case, among the first to fourth reference phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, the reference phase closest to the depth phase TOF_PH of 108 degrees is 90 degrees. Therefore, the offset calculation circuit 150 can generate a phase offset signal OFF_PH corresponding to the phase difference (i.e., 18 degrees) between the selected reference phase of 90 degrees and the depth phase TOF_PH of 108 degrees. Thereafter, the drive control circuit 130 can minimize the depth error by delaying the initial phase of the phase control signal PCTRL according to the phase offset signal OFF_PH and setting the phase difference between the reflected signal RS and the phase control signal PCTRL to 90 degrees.

[0043] Also, in the embodiment of the present invention, when the phase offset signal OFF_PH is input, the image processor 140 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH and calculate the depth information TOF_OUT corresponding to the compensated depth phase TOF_PH. For example, as shown in FIG. 6B, when the image sensing device 100 can measure up to 0 to 1500 mm, the depths corresponding to the first to fourth reference phases of 0 degrees, 90 degrees, 180 degrees, and 270 degrees can be 0 mm, 375 mm, 750 mm, and 1125 mm, respectively. The image processor 140 can reflect the phase offset signal OFF_PH on the measured depth phase TOF_PH (ideally 90 degrees) and calculate 450 mm (i.e., 1500×108 / 360) as the depth information TOF_OUT corresponding to 108 degrees.

[0044] FIG. 7 is a timing diagram for explaining the operation of the image sensing device 100 of FIG. 2 according to the first embodiment of the present invention.

[0045] As shown in FIG. 7, if the optical transmitter 110 irradiates the subject 200 with a modulation signal MS that toggles at a predetermined period, the modulation signal MS can be received as a reflected signal RS after being reflected by the subject 200. At this time, the modulation signal MS and the reflected signal RS can have a phase difference φ corresponding to the time of flight.

[0046] The phase conversion circuit 134 can generate phase information PFC of 0 degrees, 90 degrees, 180 degrees, and 270 degrees of the modulation signal MS by shifting the phase of the modulation signal MS at 90-degree intervals. The phase conversion circuit 134 can change the phase information PFC so that the phase of the phase control signal PCTRL is delayed according to the phase offset signal OFF_PH. For example, when the phase offset signal OFF_PH is calculated to be 18 degrees, the phase conversion circuit 134 can change the phase information PFC so that the phase of the phase control signal PCTRL is delayed by 18 degrees.

[0047] The sensor driving unit 138 can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS based on the phase information PFC. The sensor driving unit 138 can sequentially generate the first and second phase transition signals MIXA and MIXB corresponding to 0 degrees and 180 degrees of the modulation signal MS, and the first and second phase transition signals MIXA and MIXB corresponding to 90 degrees and 270 degrees of the modulation signal MS to drive the optical receiver 120. At this time, the sensor driving unit 138 can delay the initial phases of the first and second phase transition signals MIXA and MIXB by a delay time Td corresponding to the phase offset signal OFF_PH. As a result, the phase difference between the reflected signal RS and the phase control signal PCTRL can be set to one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Consequently, the intervals occupied by the amounts Q1, Q2, Q3, Q4 of the reflected signal RS in the activation interval of the phase control signal PCTRL are 50:50 or 100:0. Therefore, the signal-to-noise ratio (SNR) can be maximized and the depth error can be minimized.

[0048] The image processor 140 can calculate the depth phase TOF_PH based on the first and second pixel signals PIX1 and PIX2 provided from the unit pixel PX, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH. If the phase offset signal OFF_PH is input, the image processor 140 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH, and calculate the depth information TOF_OUT corresponding to the compensated depth phase TOF_PH.

[0049] Hereinafter, the operation of the image sensing device 100 according to the embodiment of the present invention will be described with reference to FIGS. 2 to 8.

[0050] FIG. 8 is a sequence diagram for explaining the operation of the image sensing device 100 according to the embodiment of the present invention.

[0051] As shown in FIG. 8, first, the depth phase TOF_PH fed back from the image processor 140 in the default state that does not reflect the phase offset signal OFF_PH can be received.

[0052] More specifically, the drive control circuit 130 generates a modulation control signal MC for adjusting the period of the modulation signal MS in response to the request RQ, and generates a phase control signal PCTRL corresponding to the phase of the low control signal RCTRL and the modulation signal MS (S810). The optical transmitter 110 irradiates the subject 200 with the modulation signal MS in response to the modulation control signal MC (S820). The pixel array 121 of the optical receiver 120 generates a pixel signal PIXs corresponding to the reflected signal RS reflected from the subject 200 in response to the low control signal RCTRL and the phase control signal PCTRL (S830). The image processor 140 calculates the depth phase TOF_PH based on the pixel signal PIXs (S840).

[0053] When the phase offset signal OFF_PH is not set (NO in S850), the offset calculation circuit 150 can calculate the phase offset signal OFF_PH based on the depth phase TOF_PH fed back from the image processor 140 (S860). That is, the offset calculation circuit 150 sets a plurality of reference phases (for example, 0 degrees, 90 degrees, 180 degrees, or 270 degrees), selects the reference phase closest to the depth phase TOF_PH among the set reference phases, and generates a phase offset signal OFF_PH corresponding to the phase difference between the selected reference phase and the depth phase TOF_PH. The drive control circuit 130 can adjust the phase of the phase control signal PCTRL in response to the phase offset signal OFF_PH (S870).

[0054] Thereafter, the optical transmitter 110 irradiates the subject 200 with the modulation signal MS in response to the modulation control signal MC (S820), and the pixel array 121 generates a pixel signal PIXs corresponding to the reflected signal RS reflected from the subject 200 in response to the low control signal RCTRL and the phase control signal PCTRL (S830). At this time, since the phase difference between the reflected signal RS and the phase control signal PCTRL is set to one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees, the signal-to-noise ratio (SNR) can be maximized and the depth error can be minimized.

[0055] Thereafter, the image processor 140 calculates a depth phase TOF_PH based on the pixel signal PIXs (S840). When the phase offset signal OFF_PH is set (YES in S850), the image processor 140 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH and calculate depth information TOF_OUT corresponding to the compensated depth phase TOF_PH (S880).

[0056] On the other hand, in the above first embodiment, the point of delaying the phase of the phase control signal PCTRL using the phase offset signal OFF_PH has been described. However, the proposed invention is not limited to this. Hereinafter, with reference to FIGS. 9 and 10, the point of adjusting the phase of the modulation signal MS using the phase offset signal OFF_PH will be described.

[0057] FIG. 9 is a block diagram for explaining a TOF type image sensing device 300 according to a second embodiment of the present invention.

[0058] As shown in FIG. 9, the image sensing device 300 can include an optical transmitter 310, an optical receiver 320, a drive control circuit 330, an image processor 340, and an offset calculation circuit 350. Since the optical transmitter 310, the optical receiver 320, the image processor 340, and the offset calculation circuit 350 in FIG. 9 are substantially the same as the configurations and the like in FIG. 2, detailed descriptions thereof are omitted.

[0059] The drive control circuit 330 can generate a modulation control signal MC for adjusting the period of the modulation signal MS in response to a request RQ, and can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS. The drive control circuit 330 can set an initial phase offset between the modulation signal MS and the phase control signal PCTRL according to the phase offset signal OFF_PH. At this time, the drive control circuit 330 according to the second embodiment can generate the modulation control signal MC so that the initial phase of the modulation signal MS is adjusted (for example, delayed) according to the phase offset signal OFF_PH. Thereby, the phase difference between the reflected signal RS and the phase control signal PCTRL can be set to one of the target values of 0 degrees, 90 degrees, 180 degrees, or 270 degrees.

[0060] According to an embodiment, the drive control circuit 330 can include a modulation controller 332, a phase conversion circuit 334, a transmission drive unit 336, and a sensor drive unit 338.

[0061] The modulation controller 332 can output a first drive control signal DRV_M so that the optical transmitter 310 can generate a modulation signal MS having a predetermined period that can be distinguished from ambient light in response to an external request RQ. The modulation controller 332 can provide the first drive control signal DRV_M to the phase conversion circuit 334. In particular, in the second embodiment, the modulation controller 332 can adjust the first drive control signal DRV_M according to the phase offset signal OFF_PH and generate an adjusted first drive control signal DRV_MD. For example, the modulation controller 332 can delay the first drive control signal DRV_M according to the phase offset signal OFF_PH and generate a delayed first drive control signal DRV_MD. The modulation controller 332 can provide the adjusted first drive control signal DRV_MD to the transmission drive unit 336.

[0062] The transmission drive unit 336 can generate a modulation control signal MC so that the initial phase of the modulation signal MS is delayed according to the adjusted first drive control signal DRV_MD. Thereby, the phase difference between the reflected signal RS and the phase control signal PCTRL can be set to a target value.

[0063] The phase conversion circuit 334 can generate a second drive control signal DRV_D so that the sensor drive unit 338 operates in conjunction with the operation timing of the transmission drive unit 336 that operates according to the first drive control signal DRV_M. Based on the first drive control signal DRV_M, the phase conversion circuit 334 changes (for example, shifts) the phase of the modulation signal MS to generate phase information PFC having different phases (for example, 0 degrees, 90 degrees, 180 degrees, or 270 degrees of the modulation signal MS (where the initial phase is adjusted or not delayed)), and can output the generated phase information PFC included in the second drive control signal DRV_D.

[0064] The sensor drive unit 338 can generate a low control signal RCTRL and a phase control signal PCTRL for controlling the pixel array 321 according to the second drive control signal DRV_D.

[0065] FIG. 10 is a timing diagram for explaining the operation of the image sensing device 300 of FIG. 9 according to the second embodiment of the present invention.

[0066] As shown in FIG. 10, the drive control circuit 330 can generate a modulation control signal MC so that the initial phase of the modulation signal MS is adjusted (for example, delayed) according to the phase offset signal OFF_PH. Thereby, although the optical transmitter 310 toggles at a predetermined period, it can irradiate the subject with the modulation signal MS whose initial phase is delayed by the delay time Td corresponding to the phase offset signal OFF_PH. After being reflected by the subject, the modulation signal MS can be received as a reflection signal RS. At this time, the modulation signal MS and the reflection signal RS can have a phase difference φ corresponding to the flight time.

[0067] The phase conversion circuit 334 can shift the phase of the modulation signal MS at 90-degree intervals to generate phase information PFC of 0 degrees, 90 degrees, 180 degrees, and 270 degrees of the modulation signal MS.

[0068] The sensor driving unit 338 can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS based on the phase information PFC. The sensor driving unit 338 can sequentially generate the first and second phase transition signals MIXA, MIXB corresponding to 0 degrees and 180 degrees of the modulation signal MS, and the first and second phase transition signals MIXA, MIXB corresponding to 90 degrees and 270 degrees of the modulation signal MS to drive the optical receiver 320.

[0069] At this time, when the initial phase of the modulation signal MS is delayed, the initial phase of the reflected signal RS is also delayed. As a result, the phase difference between the reflected signal RS and the phase control signal PCTRL can be set to one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Thereby, the intervals occupied by the amounts Q1, Q2, Q3, Q4 of the reflected signal RS in the activation interval of the phase control signal PCTRL become 50:50 or 100:0, so that the signal-to-noise ratio (SNR) can be maximized and the depth error can be minimized.

[0070] The image processor 340 can calculate the depth phase TOF_PH based on the first and second pixel signals PIX1, PIX2 provided from the unit pixel PX, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH. When the phase offset signal OFF_PH is input, the image processor 340 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH and calculate the depth information TOF_OUT corresponding to the compensated depth phase TOF_PH.

[0071] Hereinafter, with reference to FIGS. 11 and 12, the point of adjusting the phases of both the modulation signal MS and the phase control signal PCTRL using the phase offset signal OFF_PH will be described.

[0072] FIG. 11 is a block diagram for explaining a TOF type image sensing device 400 according to the third embodiment of the present invention.

[0073] As shown in FIG. 11, image sensing device 400 can include optical transmitter 410, optical receiver 420, drive control circuit 430, image processor 440, and offset calculation circuit 450. Since optical transmitter 410, optical receiver 420, image processor 440, and offset calculation circuit 450 in FIG. 11 are substantially the same as the configuration and the like in FIG. 2, detailed description thereof will be omitted.

[0074] Drive control circuit 430 can generate modulation control signal MC for adjusting the period of modulation signal MS in response to request RQ, and can generate phase control signal PCTRL corresponding to the phase of modulation signal MS. Drive control circuit 430 can set the initial phase offset between modulation signal MS and phase control signal PCTRL according to phase offset signal OFF_PH. At this time, drive control circuit 430 according to the third embodiment can be controlled to adjust (for example, delay) the initial phases of both modulation signal MS and phase control signal PCTRL according to phase offset signal OFF_PH. Thereby, the phase difference between reflection signal RS and phase control signal PCTRL can be set to one of target values of 0 degrees, 90 degrees, 180 degrees, or 270 degrees.

[0075] According to an embodiment, drive control circuit 430 can include modulation controller 432, phase conversion circuit 434, transmission drive unit 436, and sensor drive unit 438.

[0076] Modulation controller 432 can output first drive control signal DRV_MD so as to be able to generate modulation signal MS having a predetermined period that can be distinguished from ambient light by optical transmitter 410 in response to external request RQ. At this time, in the third embodiment, modulation controller 432 can change first drive control signal DRV_MD so that the initial phase of modulation signal MS is delayed according to phase offset signal OFF_PH. Modulation controller 432 can provide first drive control signal DRV_MD to transmission drive unit 436 and phase conversion circuit 434.

[0077] The transmission drive unit 436 can generate a modulation control signal MC such that the initial phase of the modulation signal MS is delayed by a first delay time Td1 in accordance with the first drive control signal DRV_MD.

[0078] The phase conversion circuit 434 can generate a second drive control signal DRV_DD in accordance with the first drive control signal DRV_MD. The phase conversion circuit 434 can change (e.g., shift) the phase of the modulation signal MS based on the first drive control signal DRV_MD to generate phase information PFC having different phases (e.g., 0 degrees, 90 degrees, 180 degrees, or 270 degrees) from each other, and can output the generated phase information PFC included in the second drive control signal DRV_DD. In particular, in the third embodiment, the phase conversion circuit 434 can change the phase information PFC such that the initial phase of the phase control signal PCTRL is delayed by a second delay time Td2 in accordance with the phase offset signal OFF_PH.

[0079] The sensor drive unit 438 can generate a low control signal RCTRL and a phase control signal PCTRL in accordance with the second drive control signal DRV_DD. The sensor drive unit 438 can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS based on the phase information PFC included in the second drive control signal DRV_DD. At this time, since the phase information PFC is changed in accordance with the phase offset signal OFF_PH, the initial phase of the phase control signal PCTRL can be delayed by the second delay time Td2 compared to the modulation signal MS.

[0080] In the third embodiment, the difference between the delayed initial phase of the modulation signal MS and the delayed initial phase of the phase control signal PCTRL can respectively correspond to the delayed initial phase of the phase control signal PCTRL in the first embodiment or the delayed initial phase of the modulation signal MS in the second embodiment.

[0081] FIG. 12 is a timing diagram for explaining the operation of the image sensing device 400 of FIG. 11 according to the third embodiment of the present invention.

[0082] As shown in FIG. 12, the drive control circuit 430 can generate a modulation control signal MC such that the initial phase of the modulation signal MS is adjusted (e.g., delayed) according to the phase offset signal OFF_PH. Thereby, although the optical transmitter 410 toggles at a predetermined period, it can irradiate the subject with the modulation signal MS whose initial phase is delayed by the first delay time Td1 corresponding to the phase offset signal OFF_PH. After being reflected by the subject, the modulation signal MS can be received as a reflection signal RS. At this time, the modulation signal MS and the reflection signal RS can have a phase difference φ corresponding to the flight time.

[0083] The phase conversion circuit 434 can generate phase information PFC of 0 degrees, 90 degrees, 180 degrees, and 270 degrees of the modulation signal MS by shifting the phase of the modulation signal MS at 90-degree intervals. The phase conversion circuit 434 can change the phase information PFC so that the initial phase of the phase control signal PCTRL is delayed by the second delay time Td2 in which the phase offset signal OFF_PH is reflected.

[0084] The sensor drive unit 438 can generate a phase control signal PCTRL corresponding to the phase of the modulation signal MS based on the phase information PFC. The sensor drive unit 438 can sequentially generate the first and second phase transition signals MIXA and MIXB corresponding to 0 degrees and 180 degrees of the modulation signal MS, and the first and second phase transition signals MIXA and MIXB corresponding to 90 degrees and 270 degrees of the modulation signal MS to drive the optical receiver 420. At this time, the sensor drive unit 438 can delay the initial phases of the first and second phase transition signals MIXA and MIXB by the second delay time Td2 corresponding to the phase offset signal OFF_PH.

[0085] At this time, the first delay time Td1 can be defined as the coarse delay time, and the second delay time Td2 can be defined as the fine delay time. That is, after roughly adjusting the initial phase of the modulation signal MS using the first delay time Td1, the initial phase of the phase control signal PCTRL can be finely adjusted using the second delay time Td2. The difference between the first delay time Td1 and the second delay time Td2 can correspond to the delay time Td in the first or second embodiment. Thereby, the phase difference between the reflected signal RS and the phase control signal PCTRL can be set to one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees. As a result, the intervals occupied by the amounts Q1, Q2, Q3, Q4 of the reflected signal RS in the activation interval of the phase control signal PCTRL become 50:50 or 100:0. Therefore, the signal-to-noise ratio (SNR) can be maximized and the depth error can be minimized.

[0086] The image processor 440 can calculate the depth phase TOF_PH based on the first and second pixel signals PIX1 and PIX2 provided from the unit pixel PX, and generate depth information TOF_OUT corresponding to the depth phase TOF_PH. When the phase offset signal OFF_PH is input, the image processor 440 can compensate the depth phase TOF_PH according to the phase offset signal OFF_PH, and calculate the depth information TOF_OUT corresponding to the compensated depth phase TOF_PH.

[0087] According to such an embodiment of the present invention, in an image sensing device that acquires depth information using the TOF method, by setting an initial phase offset between a modulation signal and a phase control signal based on the acquired depth information, the phase difference between the reflected signal and the phase control signal can be set as a phase at which the depth error is minimized. Further, by finally compensating the depth information acquired using the set initial phase offset, there is an effect that the depth error can be minimized and accurate depth information can be acquired.

[0088] Although the technical idea of the present invention has been specifically described by the above embodiments, it should be noted that the embodiments described above are for the purpose of explanation and not for limitation. Also, those of ordinary skill in the technical field of the present invention will be able to understand that various embodiments are possible through various substitutions, modifications, and changes within the scope of the technical idea of the present invention.

Claims

1. An optical transmitter that irradiates a modulated signal onto a subject, an optical receiver that generates a pixel signal corresponding to a reflected signal reflected from the subject in response to a phase control signal, an image processor that calculates a depth phase based on the pixel signal and compensates the depth phase according to a phase offset signal and outputs it as depth information, a drive control circuit that generates the phase control signal corresponding to the phase of the modulation signal and sets an initial phase difference between the phase control signal and the modulation signal according to the phase offset signal, an offset calculation circuit that sets a plurality of reference phases, selects the reference phase closest to the depth phase among the set reference phases, and calculates the phase offset signal corresponding to the phase difference between the selected reference phase and the depth phase, An image sensing device comprising the above.

2. The plurality of reference phases include a 0-degree phase, a 90-degree phase, a 180-degree phase, and a 270-degree phase. The image sensing device according to claim 1.

3. The drive control circuit delays the initial phase of the phase control signal according to the phase offset signal. The image sensing device according to claim 1.

4. The drive control circuit a modulation controller that generates a first drive control signal in response to an external request so that the optical transmitter can generate the modulation signal distinguishable from ambient light, a phase conversion circuit that generates phase information for generating the phase control signal by changing the phase of the modulation signal based on the first drive control signal and generates the phase offset signal so that the initial phase of the phase control signal is delayed, The image sensing device according to claim 1 comprising the above.

5. The drive control circuit delays the initial phase of the modulation signal according to the phase offset signal. The image sensing device according to claim 1.

6. The drive control circuit a modulation controller that generates a first drive control signal in response to an external request so that the optical transmitter can generate the modulation signal distinguishable from ambient light and adjusts the first drive control signal so that the initial phase of the modulation signal is delayed according to the phase offset signal, a phase conversion circuit that generates phase information for generating the phase control signal by changing the phase of the modulation signal based on the first drive control signal, The image sensing device according to claim 1, comprising

7. The drive control circuit The image sensing device according to claim 1, wherein the drive control circuit delays both the initial phases of the modulation signal and the phase control signal according to the phase offset signal.

8. The drive control circuit In response to an external request, a modulation controller that generates a first drive control signal so that the optical transmitter can generate the modulation signal distinguishable from ambient light, and adjusts the first drive control signal so that the initial phase of the modulation signal is delayed according to the phase offset signal; Based on the first drive control signal, a phase conversion circuit that changes the phase of the modulation signal to generate phase information for generating the phase control signal, and changes the phase information so that the initial phase of the phase control signal is delayed according to the phase offset signal; The image sensing device according to claim 1, comprising

9. The image processor A signal conversion circuit that converts the pixel signal into digital-form data; A signal processing circuit that performs a predetermined arithmetic process on the data to calculate the depth phase, and calculates the depth information by reflecting the phase offset signal in the depth phase; The image sensing device according to claim 1, comprising

10. The optical receiver Includes a pixel array in which a plurality of unit pixels are arranged in an array form, The image sensing device according to claim 1, wherein each unit pixel includes at least two pixels that operate at different phases of the phase control signal.

11. Irradiating the subject with a modulation signal by the optical receiver; Generating, by the optical transmitter, a pixel signal corresponding to a reflection signal reflected from the subject according to the phase control signal; Generating a depth phase based on the pixel signal; Setting a plurality of reference phases, selecting, among the set reference phases, the reference phase closest to the depth phase, and calculating a phase offset signal corresponding to the phase difference between the selected reference phase and the depth phase; Setting an initial phase difference between the phase control signal and the modulation signal according to the phase offset signal; Compensating the depth phase according to the phase offset signal and outputting the compensated depth phase as depth information; An image sensing method including

12. The plurality of reference phases are The image sensing method according to claim 11, which is set to a phase of 0 degrees, a phase of 90 degrees, a phase of 180 degrees, or a phase of 270 degrees.

13. The step of setting the initial phase difference is The image sensing method according to claim 11, wherein the initial phase of the phase control signal is delayed according to the phase offset signal.

14. The step of setting the initial phase difference is The image sensing method according to claim 11, wherein the initial phase of the modulation signal is delayed according to the phase offset signal.

15. The step of setting the initial phase difference is The image sensing method according to claim 11, wherein the initial phases of the modulation signal and the phase control signal are both delayed according to the phase offset signal.

16. The step of compensating the depth phase and outputting it as depth information is The step of converting the pixel signal into digital form data, Performing a predetermined calculation process on the data to calculate the depth phase, and reflecting the phase offset signal in the depth phase to calculate the depth information, The image sensing method according to claim 11, comprising:

17. The optical transmitter includes a pixel array in which a plurality of unit pixels are arranged in an array form, The image sensing method according to claim 11, wherein each unit pixel includes at least two pixels operating in different phases of the phase control signal.

18. Generating a source signal and irradiating a target, Receiving a reflection signal, which is the source signal reflected from the target, Generating a phase control signal having a phase (lagging phase) delayed by a specific amount specified according to the phase offset signal with respect to the source signal, Controlling a unit pixel via the phase control signal and generating a pixel signal based on the reflection signal, Calculating a depth phase according to the pixel signal, Compensating the depth phase according to the phase offset signal to generate depth information, Selecting a reference phase closest to the depth phase among a plurality of reference phases, Generating the phase offset signal according to the phase difference between the selected reference phase and the depth phase, A driving method of an image sensing device, comprising:

Images