Time-of-flight sensor, time-of-flight system, and imaging system

Abstract

A time-of-flight device includes a pixel array, and the pixel array includes a pixel circuit array. Among them, the columns of the array include: a first pixel circuit, the first pixel circuit including a first photodiode, a first capacitor and a second capacitor coupled to the first photodiode, and a second pixel circuit, the second pixel circuit including a second photodiode, a third capacitor and a fourth capacitor coupled to the second photodiode, a first signal line coupled to the first capacitor; a second signal line coupled to the second capacitor; a third signal line coupled to the third capacitor; a fourth signal line coupled to the fourth capacitor, a first switch circuit, a second switch circuit, a first comparator coupled to the first signal line and the third signal line through the first switch circuit, and a second comparator coupled to the second signal line and the fourth signal line through the second switch circuit.

Description

Technical Field

[0001] The present application generally relates to image sensors. More specifically, the present application relates to systems and methods for distance and depth determination in time-of-flight (TOF) sensors, time-of-flight systems, and imaging systems. Background Art

[0002] Image sensing devices generally include an image sensor typically implemented as an array of pixel circuits, as well as signal processing circuitry and any associated control or timing circuitry. Within the image sensor itself, charge is collected in the photoelectric conversion device of the pixel circuit due to the impact of light. There are typically a very large number of individual photoelectric conversion devices (e.g., tens of millions), as well as many signal processing circuit components operating in parallel. Various components within the signal processing circuit are shared by a large number of photoelectric conversion devices; for example, one or more columns of photoelectric conversion devices may share a single analog-to-digital converter (ADC) or sample-and-hold (S / H) circuit. Summary of the Invention

[0003] [Technical Problem]

[0004] Aspects of the present disclosure relate to image sensors and methods for distance determination therein.

[0005] [Solution to the Problem]

[0006] In one aspect of the present disclosure, a time-of-flight sensor is provided, comprising: a pixel array including a plurality of pixel circuits arranged in an array, wherein a first column of the array includes: a first pixel circuit including a first photodiode, a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode; and a second pixel circuit including a second photodiode, a third capacitor coupled to the second photodiode, and a fourth capacitor coupled to the second photodiode; a first signal line coupled to the first capacitor; a second signal line coupled to the second capacitor; a third signal line coupled to the third capacitor; a fourth signal line coupled to the fourth capacitor; a first switch circuit; a second switch circuit; a first comparator coupled to the first signal line and the third signal line through the first switch circuit; and a second comparator coupled to the second signal line and the fourth signal line through the second switch circuit.

[0007] In another aspect of the present disclosure, a time-of-flight system is provided, including: a light source configured to emit light; and a sensor including: a pixel array including a plurality of pixel circuits arranged in an array, wherein columns of the array include: a first pixel circuit including a first photodiode, a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode, and a second pixel circuit including: a second photodiode; a third capacitor coupled to the second photodiode; and a fourth capacitor coupled to the second photodiode; a first signal line coupled to the first capacitor, a second signal line coupled to the second capacitor, a third signal line coupled to the third capacitor, a fourth signal line coupled to the fourth capacitor, a first switching circuit, a second switching circuit; a first comparator coupled to the first signal line and the third signal line through the first switching circuit; and a second comparator coupled to the second signal line and the fourth signal line through the second switching circuit.

[0008] In another aspect of the present disclosure, a system is provided, including: a first sensor configured to generate image data, the first sensor including a first pixel array; and a second sensor configured to generate distance data, the second sensor including: a second pixel array including a plurality of pixel circuits arranged in an array, wherein columns of the array include: a first pixel circuit including a first photodiode; a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode, and a second pixel circuit including a second photodiode, a third capacitor coupled to the second photodiode, and a fourth capacitor coupled to the second photodiode, a first signal line coupled to the first capacitor, a second signal line coupled to the second capacitor, a third signal line coupled to the third capacitor, a fourth signal line coupled to the fourth capacitor; a first switching circuit; a second switching circuit; a first comparator coupled to the first signal line and the third signal line through the first switching circuit, and a second comparator coupled to the second signal line and the fourth signal line through the second switching circuit.

[0009] Accordingly, various aspects of the present disclosure provide improvements at least in the technical field of depth sensing and related technical fields such as imaging and image processing.

[0010] The present disclosure may be embodied in various forms, including computer-implemented methods, computer program products, computer systems and networks, hardware or circuits controlled by user interfaces and application programming interfaces; and hardware-implemented methods, signal processing circuits, image sensor circuits, application-specific integrated circuits, field-programmable gate arrays, and the like. The foregoing summary is only intended to give a general idea of the various aspects of the present disclosure and does not limit the scope of the present disclosure in any way.

[0011] [Advantageous Effects of the Present Invention]

[0012] The quantities I and Q can be obtained in the Nth frame, and the quantities -I and -Q can be obtained in the (N + 1)th frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Reference is made to the accompanying drawings, in which these and other more detailed and specific features of various embodiments are disclosed more fully in the following description, wherein:

[0014] Figure 1A Figure 1A An exemplary TOF system according to various aspects of the present disclosure is shown;

[0015] Figure 1B Figure 1B An exemplary TOF system according to various aspects of the present disclosure is shown;

[0016] Figure 2 Figure 2 An exemplary TOF sensor according to various aspects of the present disclosure is shown;

[0017] Figure 3A Figure 3A An exemplary pixel circuit according to various aspects of the present disclosure is shown;

[0018] Figure 3B Figure 3B An exemplary pixel circuit according to various aspects of the present disclosure is shown;

[0019] Figure 4 Figure 4 An exemplary readout circuit according to various aspects of the present disclosure is shown;

[0020] Figure 5A Figure 5A Shows Figure 4 exemplary readout modes and operations in the exemplary readout circuit;

[0021] Figure 5B Figure 5B Shows Figure 4 exemplary readout modes and operations in the exemplary readout circuit;

[0022] Figure 5C Figure 5C Shows Figure 4 exemplary readout modes and operations in the exemplary readout circuit;

[0023] Figure 5D Figure 5D Shows Figure 4 exemplary readout modes and operations in the exemplary readout circuit;​​​​​​​​​​​​​​​​​​​​

[0024] Figure 6 Figure 6 Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0025] Figure 7 Figure 7 Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0026] Figure 8 Figure 8 Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0027] Figure 9A Figure 9A Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0028] Figure 9B Figure 9B Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0029] Figure 10 Figure 10 Shows the exemplary IQ mosaic pattern used in connection with various aspects of the present disclosure;

[0030] Figure 11A Figure 11A Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0031] Figure 11B Figure 11B Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0032] Figure 12A Figure 12A Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0033] Figure 12B Figure 12B Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ;

[0034] Figure 13A Figure 13A Shows the exemplary readout mode and operation in the exemplary readout circuit of Figure 4 ; ​​​​​​​​​​​​​​​​​​​​​​

[0035] Figure 13B Figure 13B illustrates Figure 4 exemplary readout patterns and operations in an exemplary readout circuit of;

[0036] Figure 14A Figure 14A illustrates Figure 4 exemplary readout patterns and operations in an exemplary readout circuit of;

[0037] Figure 14B Figure 14B illustrates Figure 4 exemplary readout patterns and operations in an exemplary readout circuit of;

[0038] Figure 15 Figure 15 illustrates an exemplary operation method according to various aspects of the present disclosure; and

[0039] Figure 16 Figure 16 illustrates another exemplary operation method according to various aspects of the present disclosure. DETAILED DESCRIPTION

[0040] In the following description, numerous details are set forth, such as flowcharts, data tables, and system configurations. It will be apparent to those skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of the present application.

[0041] Furthermore, while the present disclosure mainly focuses on examples where the processing circuit is used in an image sensor, it will be understood that this is merely one example of an implementation. It will be further understood that the disclosed systems and methods can be used in any device where distance detection in a wave-based sensor is required; for example, audio circuits, phonon sensors, radar systems, etc.

[0042] Imaging system

[0043] Figure 1A ​​​​​​​​​​A first example 100a of a TOF imaging system 101a configured to detect and / or image an object 102 located at a distance d is shown. The TOF imaging system 101a includes a light generator 111 configured to generate an emitted light wave 120 toward the object 102 and a TOF image sensor 112 configured to receive a reflected light wave 130 from the object 102. The emitted light wave 120 may have a periodic waveform. The TOF image sensor 112 can be any device capable of converting incident radiation into a signal. For example, the TOF image sensor 112 can be implemented by a complementary metal-oxide-semiconductor (CMOS) image sensor (CIS), a charge-coupled device (CCD), etc. The TOF imaging system 101a may further include a distance determination circuit, such as a controller 113 (e.g., a CPU) and a memory 114, which are operable to perform one or more examples of time-of-flight processing as further described below.

[0044] Figure 1B A second example 100b of a TOF imaging system 101b configured to detect and / or image an object 102 located at a distance d is shown. The TOF imaging system 101b includes a light generator 111 configured to generate an emitted light wave 120 toward the object 102, a TOF image sensor 112 configured to receive a reflected light wave 130 from the object 102, and an RGB image sensor 115 configured to capture an RGB image of the object 102. The emitted light wave 120 may have a periodic waveform. The TOF image sensor 112 can be any device capable of converting incident radiation into a signal. For example, the TOF image sensor 112 and the RGB sensor 115 can each be implemented by a complementary metal-oxide-semiconductor (CMOS) image sensor (CIS), a charge-coupled device (CCD), etc. Although the second example 100b describes an RGB image sensor 115, in reality, the image sensor 115 can capture a monochromatic image or can include color filters different from RGB. Additionally, although Figure 1B the TOF image sensor 112 and the RGB image sensor 115 are illustrated as separate components, in some aspects of the present disclosure, the TOF image sensor 112 and the RGB image sensor 115 can be integrated into a single chip and / or utilize a single pixel array. The TOF imaging system 101b may further include a distance determination and processing circuit, such as a controller 113 (e.g., a CPU) and a memory 114, which are operable to perform one or more examples of time-of-flight and image processing as further described below.

[0045] For example, the light generator 111 can be a light-emitting diode (LED), a laser diode, or any other light-generating device or combination of devices, and the light waveform can be controlled by the controller 113. The light generator can operate in the infrared range to reduce interference from the visible spectrum, although any wavelength range that can be sensed by the image sensor 112 can be utilized. The controller 113 can be configured to receive an image from the image sensor and calculate a depth map indicating the distance d to each point of the object 102.

[0046] Figure 2 An exemplary image sensor 200 in accordance with various aspects of the present disclosure is shown. The image sensor 200 can be an example of the TOF image sensor 112 shown in FIG. 1. As Figure 2 shown, the image sensor 200 includes an array 210 of pixel circuits 211, each pixel circuit being located at an intersection where a horizontal signal line 212 intersects a set of vertical signal lines 213a, 213b, 213c, 213d. The horizontal signal line 212 is operatively connected to a vertical scan circuit 220 (also referred to as a "row scan circuit" or "vertical drive circuit") at a point outside the pixel array 210. The horizontal signal line 212 transmits signals from the vertical scan circuit 220 to a particular row of pixel circuits 211. Although Figure 2 a single horizontal signal line 212 is shown for the pixel circuits 211 of a given row, in practice, multiple horizontal signal lines 212 can be provided for each row of pixel circuits 211.

[0047] The pixel circuit 211 alternately stores charges corresponding to the amount of incident light in floating diffusions FDa and FDb (e.g., as shown in FIG. 3), and selectively outputs an analog signal corresponding to the amount of charge to the vertical signal lines 213a, 213b, 213c, 213d in a manner that will be described in more detail below. Although Figure 2 vertical signal lines 213a and 213c are shown on one side of a given pixel circuit 211 and vertical signal lines 213b and 213d are shown on the other side of the given pixel circuit 211, in practice, the vertical signal lines 213a, 213b, 213c, 213d can all be provided on a single side of the given pixel circuit 211; or one of the vertical signal lines 213a, 213b, 213c, 213d can be located on one side of the given pixel circuit 211 and the other three of the vertical signal lines 213a, 213b, 213c, 213d can be located on the other side of the given pixel circuit 211. Additionally, for illustrative purposes, Figure 2 only a subset of the pixel circuits 211 in the array 210 is actually shown; however, in practice, the image sensor 200 can have any number of pixel circuits 211. Figure 2Shows two vertical signal lines 213a and 213b or 213c and 213d of each pixel circuit 211 (“dual-head” system); however, in practice, the image sensor 200 may include a greater number of vertical signal lines for each column of pixel circuits 211.

[0048] The pixel circuits 211 in some rows of the array 210 are connected to the vertical signal lines 213a and 213b, while the pixel circuits 211 in other rows of the array 210 are connected to the vertical signal lines 213c and 213d. In some aspects, the pixel circuits 211 are connected to specific vertical signal lines in a group of four rows, i.e., the pixel circuits 211 in the first four rows of the array 210 are connected to the vertical signal lines 213a and 213b, the pixel circuits 211 in the second four rows of the array 210 are connected to the vertical signal lines 213c and 213d, the pixel circuits in the third four rows of the array 210 are connected to the vertical signal lines 213c and 213d, and so on.

[0049] The vertical signal lines 213a, 213b, 213c, 213d conduct analog signals (A for the vertical signal lines 213a and 213c and B for the vertical signal lines 213b and 213d) for a specific column to the readout circuit 231, which includes a switch circuit 232 and includes two comparators 234 for each column of the pixel circuits 211. Each comparator 234 compares the analog signal with a reference signal output from a reference signal generator 233. The reference signal generator 233 can be, for example, a digital-to-analog converter (DAC), and the reference signal can have, for example, a periodic ramp waveform. Each comparator 234 outputs a digital signal indicating the comparison between the input analog signal from the corresponding signal line input and the reference signal.

[0050] The output of the readout circuit 231 is provided to the signal processing circuit 235. The signal processing circuit 235 may include additional components such as counters, latches, S / H circuits, etc. The signal processing circuit 235 is capable of performing the method of correlated double sampling (CDS). CDS can overcome some pixel noise-related problems by sampling each pixel circuit 211 twice. First, the reset voltage of the pixel circuit 211 is sampled. This can also be referred to as the P-phase value or the cds value. Subsequently, the data voltage Vdata of the pixel circuit 211 (i.e., the voltage after the pixel circuit 211 is exposed) is sampled. This can also be referred to as the D-phase value or the exposure value. Then the reset value Vreset is subtracted from the data value Vdata to provide a value reflecting the amount of light falling on the pixel circuit 211. The CDS method can be performed for each head of the pixel circuit 211.

[0051] Various components of the signal processing circuit are controlled by the horizontal scanning circuit 240, which is also referred to as the "column scanning circuit" or "horizontal driving circuit". The horizontal scanning circuit 240 causes the signal processing circuit to output signals via the output circuit 250 for further processing, storage, transmission, etc. The vertical scanning circuit 220, the switching circuit 232, the reference circuit generator 233, and the horizontal circuit 240 can operate under the control of the drive controller 260 and / or the communication and timing circuit 270, and the drive controller 260 and / or the communication and timing circuit 270 can in turn operate based on the clock circuit 280. The clock circuit 280 can be a clock generator that generates one or more clock signals for different components of the image sensor 200. Additionally or alternatively, the clock circuit 280 can be a clock converter that converts one or more clock signals received from outside the image sensor 200 and provides the converted clock signals to different components of the image sensor 200.

[0052] Figure 3A A first exemplary pixel circuit 300a with a dual-head configuration is shown. The pixel circuit 300a can be Figure 2 an example of the pixel circuit 211 shown in the first or second row of the array 210. As Figure 3A shown, the pixel circuit 300a includes a photoelectric conversion element 301 (e.g., a photodiode), a pixel reset transistor 302, a first transfer transistor 303a, a second transfer transistor 303b, a first floating diffusion FDa, a second floating diffusion FDb, a first head reset transistor 304a, a second head reset transistor 304b, a first intervening transistor 305a, a second intervening transistor 305b, a first amplifier transistor 306a, a second amplifier transistor 306b, a first selection transistor 307a, and a second selection transistor 307b. The photoelectric conversion element 301, the first transfer transistor 303a, the first head reset transistor 304a, the first intervening transistor 305a, the first amplifier transistor 306a, and the first selection transistor 307a are controlled to output an analog signal (A) via the first vertical signal line 308a, which can be Figure 2 an example of the vertical signal line 213a shown in Figure 2 The set of components can be referred to as "head A". The photoelectric conversion element 301, the second transfer transistor 303b, the second head reset transistor 304b, the second intervening transistor 305b, the second amplifier transistor 306b, and the second selection transistor 307b are controlled to output an analog signal (B) via the second vertical signal line 308b, which can be Figure 2 an example of the vertical signal line 213b shown in Figure 3A also shown and can be Figure 2The third vertical signal line 308c, which is an example of the vertical signal line 213c shown in, and may be Figure 2 The fourth vertical signal line 308d, which is an example of the vertical signal line 213d shown in. However, as Figure 3A shown, the pixel circuit 300a is not connected to the third vertical signal line 308c or the fourth vertical signal line 308d.

[0053] Figure 3B A second exemplary pixel circuit 300b with a dual-head configuration is shown. The pixel circuit 300b may be Figure 2 An example of the pixel circuit 211 shown in the last row of the array 210 in. As Figure 3B shown, the pixel circuit 300b has a structural similarity to the Figure 3A pixel circuit 300a, and includes a photoelectric conversion element 301 (e.g., a photodiode), a pixel reset transistor 302, a first transfer transistor 303a, a second transfer transistor 303b, a first floating diffusion FDa, a second floating diffusion FDb, a first head reset transistor 304a, a second head reset transistor 304b, a first intervening transistor 305a, a second intervening transistor 305b, a first amplifier transistor 306a, a second amplifier transistor 306b, a first selection transistor 307a, and a second selection transistor 307b. The photoelectric conversion element 301, the first transfer transistor 303a, the first head reset transistor 304a, the first intermediate transistor 305a, the first amplifier transistor 306a, and the first selection transistor 307a are controlled to output an analog signal (A) via the third vertical signal line 308c. This set of components may be referred to as "head A". The photoelectric conversion element 301, the second transfer transistor 303b, the second head reset transistor 304b, the second intervening transistor 305b, the second amplifier transistor 306b, and the second selection transistor 307b are controlled to output an analog signal (B) via the fourth vertical signal line 308d. This set of components may be referred to as "head B". Figure 3B The first vertical signal line 308a and the second vertical signal line 308b are also shown. However, as Figure 3B shown, the pixel circuit 300b is not connected to the first vertical signal line 308a or the second vertical signal line 308b.

[0054] In any pixel circuit (300a or 300b), the first transfer transistor 303a and the second transfer transistor 303b are controlled by control signals on the first transfer gate line 309a and the second transfer gate line 309b, respectively. The first head reset transistor 304a and the second head reset transistor 304b are controlled by a control signal on the head reset gate line 310. The first intervening transistor 305a and the second intervening transistor 305b are controlled by a control signal on the FD gate line 311. The first selection transistor 307a and the second selection transistor 307b are controlled by a control signal on the selection gate line 312. The first transfer gate line 309a, the second transfer gate line 309b, the head reset gate line 310, the FD gate line 311, and the selection gate line 312 may be Figure 2 an example of the horizontal signal line 212 shown in

[0055] In operation, the pixel circuit 300a or the pixel circuit 300b is controlled in a time-division manner such that, during half of the horizontal period, incident light is converted via head A to generate an output signal A; and, during the other half of the horizontal period, incident light is converted via head B to generate an output signal B. The division of the frame between the head A part and the head B part may be referred to as the phase of the head. For example, in the case where the horizontal period runs from 0 to t, the pixel circuit 300a or the pixel circuit 300b may be controlled such that head A operates from 0 to t / 2 (0 phase) and head B operates from t / 2 to t (180 phase), such that head A operates from t / 4 to 3 / 4 (90 phase) and head B operates from 0 to t / 4 and from 3 / 4 to t (270 phase), such that head A operates from t / 2 to t and head B operates from 0 to t / 2, or such that head A operates from 0 to t / 4 and from 3t / 4 to t and head B operates from t / 4 to 3t / 4. Under such operation, the quantities Q and I of the pixel circuit 300a or the pixel circuit 300b may be defined such that Q is given by subtracting 180 phase from 0 phase and I is given by subtracting 270 phase from 90 phase.

[0056] Although Figures 3A to 3B the pixel circuit 300a and the pixel circuit 300b having a plurality of transistors in a specific configuration are shown, the present disclosure is not limited thereto and may be applied to configurations in which the pixel circuit 300a or the pixel circuit 300b includes fewer or more transistors and other elements, such as additional capacitors, resistors, etc.

[0057] Readout mode

[0058] The image sensor according to the present disclosure may have a plurality of different readout modes, which will be described first with reference to Figure 4 as follows. Figure 4 A part of the pixel array is shown, such as Figure 2the array 210 shown; and a part of the readout circuit, such as Figure 2 the readout circuit 231 shown. Specifically, Figure 4 pixel circuits 410 of two adjacent columns are shown, which may be the same as or similar to the pixel circuits 211 shown in Figure 2 and / or the pixel circuits 300a, 300b shown in Figure 3A and Figure 3B ; four vertical signal lines 420a, 420b, 420c, 420b per column, which may be the same as or similar to the vertical signal lines 213a, 213b, 213d shown in Figure 2 and / or the vertical signal lines 308a, 308b, 308c, 308d shown in Figure 3A and Figure 3B ; a switch circuit 430, which may be the same as or similar to the switch circuit 232 shown in Figure 2 ; a reference signal generator 440, which may be the same as or similar to the reference signal generator 233 shown in Figure 2 ; and a plurality of comparators 450, which may be the same as or similar to the comparator 234 shown in Figure 2 . Each pixel circuit 410 is shown bisected by a dashed line, thereby showing two heads of each pixel circuit 410.

[0059] The four pixel circuits 410 below are coupled to the first vertical signal line 420a at one head (e.g., at the capacitor forming the first floating diffusion FDa), and to the second vertical signal line 420b at the other head (e.g., at the capacitor forming the second floating diffusion FDb). Thus, the four pixel circuits 410 below may each correspond to the pixel circuit 300a shown in Figure 3A . The four pixel circuits 410 above are coupled to the third vertical signal line 420c at one head (e.g., at the capacitor forming the first floating diffusion FDa), and to the fourth vertical signal line 420d at the other head (e.g., at the capacitor forming the second floating diffusion FDb). Thus, the four pixel circuits 410 below may each correspond to the pixel circuit 300b shown in Figure 3B .

[0060] For a given column, the switch circuit 430 includes a first set of switch circuits 431 and a second set of switch circuits 432. As shown, each set of switch circuits 431, 432 includes three switches, and each switch can be controlled individually. The first set of switch circuits 431 includes a first switch connected at a first end to the first vertical signal line 420a of the left column, a second switch connected at a first end to the third vertical signal line 420c of the left column, and a third switch connected at a first end to the first vertical signal line 420a of the right column. The second ends of the first, second, and third switches are coupled (capacitively coupled as shown) to the first input of the first comparator 450 of the left column. The second set of switch circuits 431 includes a first switch connected at a first end to the second vertical signal line 420b of the left column, a second switch connected at a first end to the fourth vertical signal line 420d of the left column, and a third switch connected at a first end to the second vertical signal line 420b of the right column. The second ends of the first, second, and third switches are coupled (capacitively coupled as shown) to the first input of the second comparator 450 of the left column. The second input of the first comparator 450 and the second comparator 450 are coupled (capacitively coupled as shown) to the reference signal generator 440.

[0061] Accordingly, the first comparator 450 is coupled to at least the first vertical signal line 420a and the third vertical signal line 420c through the first switch circuit 431, and the second comparator 450 is coupled to at least the second vertical signal line 420b and the fourth vertical signal line 420d through the second switch circuit 432. The first switch circuit 431 and the second switch circuit 432 can be controlled by a timing circuit, such as Figure 2 the communication and timing circuit 270 shown in. The pixel circuit 411 of the left column is also coupled to the comparator 450 of the right column; for example, the third vertical signal line 420c is connected to the third switch of the first switch circuit 431 of the right column, and the fourth vertical signal line 420d is connected to the third switch of the second switch circuit 432 of the right column.

[0062] In conjunction with Figures 5A - 14B more detailed description Figure 4 the various readout modes of the readout circuit section shown in. The components shown in Figures 5A to 14B correspond to the components shown in Figure 4 and thus the detailed description of the components is not repeated.

[0063] Figures 5A to 5D The so-called normal modes of the Nth frame to the (N + 3)th frame are shown respectively. In Figure 5AIn [description], the Nth frame is shown. As shown, the first switch of each of the first switch circuit 431 and the second switch circuit 432 is closed, while the second switch and the third switch of each of the first switch circuit 431 and the second switch circuit 432 are open. The pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 operates at the 0 phase, and the head B of the corresponding pixel circuit 410 operates at the 180 phase.

[0064] In Figure 5B [description], the (N + 1)th frame is shown. As shown, the states of the first switch circuit 431 and the second switch circuit 432 are the same as those in Figure 5A [description]; however, the phases of the pixel circuits 410 are modified. Thus, the pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 operates at the 180 phase, and the head B of the corresponding pixel circuit 410 operates at the 0 phase.

[0065] In Figure 5C [description], the (N + 2)th frame is shown. As shown, the states of the first switch circuit 431 and the second switch circuit 432 are the same as those in Figure 5A [description]; however, the phases of the pixel circuits 410 are modified. Thus, the pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 operates at the 90 phase, and the head B of the corresponding pixel circuit 410 operates at the 270 phase.

[0066] In Figure 5D [description], the (N + 3)th frame is shown. As shown, the states of the first switch circuit 431 and the second switch circuit 432 are the same as those in Figure 5A [description]; however, the phases of the pixel circuits 410 are modified. Thus, the pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 operates at the 270 phase, and the head B of the corresponding pixel circuit 410 operates at the 90 phase.

[0067] The output of each frame and / or each horizontal period within a frame can be stored in a memory. After these four frames, the quantities Q and I can be calculated as described above. In some aspects of the present disclosure, the quantities Q and the quantity I are calculated in a signal processing circuit (such as the signal processing circuit 235 shown in Figure 2 [description]) provided after the comparator 450. The signal processing circuit may include a memory and a computing circuit, such as a processor (e.g., a CPU or an FPGA).

[0068] Figure 6 shows the so-called pixel thinning or pixel skipping mode for the Nth frame. Specifically, Figure 6 shows the "skip 1" mode that skips one row of pixels; however, the present disclosure can also be implemented with a "skip 2" readout mode that skips two rows of pixels. As Figure 6 shown, the first switch of each of the first switch circuit 431 and the second switch circuit 432 is closed, while the second and third switches of each of the first switch circuit 431 and the second switch circuit 432 are open. Every other pixel circuit 410 in the bottom four rows is driven in two consecutive horizontal periods 1H to 2H such that in each horizontal period, the head A of the corresponding pixel circuit 410 operates at 0 phase and the head B of the corresponding pixel circuit 410 operates at 180 phase.

[0069] In Figure 6 , the pixel circuits 410 located in the bottom row and the pixel circuits 410 located in the third row from the bottom in the frame are read out while skipping the pixel circuits 410 located in the second row from the bottom and the pixel circuits 410 located in the fourth row from the bottom. Following Figure 6 the Nth frame shown in, the phase of the pixel circuits 410 can be modified in a manner described with respect to Figures 5A to 5D the (N + 1)th to (N + 3)th frames. Thus, after these four frames, the quantities Q and I can be calculated. However, compared with Figures 5A to 5D the normal mode of, the skip 1 mode can be implemented in half the time because each frame only includes half the number of horizontal periods.

[0070] Figure 7 shows the so-called pixel merging mode for the Nth frame. Specifically, Figure 7 shows the "2×2 merge" mode that merges four pixel groups; however, the present disclosure can be implemented with a "2×4 merge mode", "1×2 merge mode", "1×4 merge mode", etc. As Figure 7 shown, the first and third switches of each of the first switch circuit 431 and the second switch circuit 432 are closed, while the second switch of each of the first switch circuit 431 and the second switch circuit 432 is open.

[0071] In the first horizontal period 1H, the first, second, fifth, and sixth rows (counting from the bottom) of the driving pixel circuit 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 0 phase and the head B of the corresponding pixel circuit 410 operates at the 180 phase. During the first horizontal period 1H, the signals for the first and second rows of the pixel circuits 410 in both the left column and the right column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the fifth and sixth rows of the pixel circuits 410 in both the left column and the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0072] In the second horizontal period 2H, the third, fourth, seventh, and eighth rows of the pixel circuit 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 0 phase and the head B of the corresponding pixel circuit 410 operates at the 180 phase. During the second horizontal period 2H, the signals for the third and fourth rows of the pixel circuits 410 in both the left column and the right column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the seventh and eighth rows of the pixel circuits 410 in both the left column and the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0073] After the Figure 7 Nth frame shown, the phase of the pixel circuit 410 can be modified in a manner described for the (N + 1)th to (N + 3)th frames. Thus, after these four frames, the quantities Q and I can be calculated. However, compared with Figures 5A to 5D the normal mode of Figures 5A to 5D the 2×2 merging mode can be implemented in half the time because each frame contains only half the number of horizontal periods.

[0074] The merging and skipping modes can be combined into a hybrid mode. Figure 8 This hybrid mode of implementing 2×4 merging and skipping 2 is shown. As Figure 8 shown, the first and third switches of the first switch circuit 431 and the second switch circuit 432 in the left column are closed, while the second switch of each of the first switch circuit 431 and the second switch circuit 432 in the left column is open. All three switches of the first switch circuit 431 and the second switch circuit 432 in the right column are open.

[0075] In the first horizontal period 1H, the bottom four rows of the drive pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 0 phase and the head B of the corresponding pixel circuit 410 operates at the 180 phase. During the first horizontal period 1H, signals for the bottom four rows of the pixel circuits 410 in both the left column and the right column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column.

[0076] Skip the next four rows of the pixel circuits 410 such that in the second horizontal period 2H, the bottom four rows of the next set of eight pixel circuits ( Figure 8 not shown in Figure 8 are driven. After the Nth frame shown in Figures 5A to 5D , the phases of the pixel circuits 410 can be modified in a manner described for the (N + 1)th to (N + 3)th frames. Thus, after these four frames, the quantities Q and I can be calculated. However, compared with Figures 5A to 5D the normal mode, the 2×4 skip 2 mode can be implemented in a quarter of the time because each frame includes only a quarter of the horizontal periods.

[0077] In each of the above modes, since four phases per pixel are utilized, four frames are used to obtain the quantities Q and I. However, in some modes, the quantities Q and I are obtained in only two frames by utilizing two phases per pixel. These modes can be referred to as IQ modes. The IQ modes can be implemented using any one of the above normal mode, skip mode, merge mode, and hybrid mode. Figures 9A - 9B The 2×2 merge IQ mode is shown.

[0078] In Figure 9A , the Nth frame is shown. As shown, the first and third switches of the first switch circuit 431 and the second switch circuit 432 in the left column and the second and third switches of the first switch circuit 431 and the second switch circuit 432 in the right column are closed, while the second switch of the first switch circuit 431 and the second switch circuit 432 in the left column and the first switch of the first switch circuit 431 and the second switch circuit 432 in the right column are open. In the first horizontal period 1H, the pixel circuits 410 in the first and third rows (counting from the bottom) are driven such that the head A of the corresponding pixel circuit 410 operates at the 0 phase and the head B of the corresponding pixel circuit 410 operates at the 180 phase, and the pixel circuits 410 in the fifth and seventh rows are driven such that the head A of the corresponding pixel circuit 410 operates at the 90 phase and the head B of the corresponding pixel circuit 410 operates at the 270 phase.

[0079] In the second horizontal period 2H, the second and fourth row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 180 phase, and the head B of the corresponding pixel circuit 410 operates at the 0 phase, and the fifth and seventh row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 270 phase, and the head B of the corresponding pixel circuit 410 operates at the 90 phase.

[0080] During the first horizontal period 1H and the second horizontal period 2H, signals for the bottom four rows of the pixel circuits 410 in both the left and right columns are provided to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column. At the same time, signals for the top four rows of the pixel circuits 410 in both the left and right columns are provided to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0081] In Figure 9B shows the (N + 1)th frame. The configuration of each switch in the first switch circuit 431 and the second switch circuit 432 remains the same as that in Figure 9A In the first horizontal period 1H, the first and third row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 90 phase, and the head B of the corresponding pixel circuit 410 operates at the 270 phase, and the fifth and seventh row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 0 phase, and the head B of the corresponding pixel circuit 410 operates at the 180 phase.

[0082] In the second horizontal period 2H, the second and fourth row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 270 phase, and the head B of the corresponding pixel circuit 410 operates at the 90 phase, and the fifth and seventh row pixel circuits 410 are driven such that the head A of the corresponding pixel circuit 410 operates at the 180 phase, and the head B of the corresponding pixel circuit 410 operates at the 0 phase.

[0083] During the first horizontal period 1H and the second horizontal period 2H, signals for the bottom four rows of the pixel circuits 410 in both the left and right columns are provided to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column. At the same time, signals for the top four rows of the pixel circuits 410 in both the left and right columns are provided to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0084] IQ mosaic / demosaic readout mode

[0085] Figure 10An exemplary mosaic / demosaic (mdm) process that can be implemented with these modes is shown and will be discussed in more detail below. As Figure 10 shown, a first data block 1010 and a second data block 1020 are obtained. The first data block 1010 and the second data block can be obtained, for example, by the process discussed with respect to Figures 11A - 14B discussion.

[0086] The first data block 1010 includes pixel data corresponding to the quantities I and Q in alternating columns. The second data block 1020 includes pixel data corresponding to the quantities -I and -Q in alternating columns. By subtracting the second data block 1020 from the first data block 1010, a third data block 1030 is obtained. The third data block 1030 includes data corresponding to the quantities Q' and I', where the quantities Q' and I' correspond to pixel data with environmental errors eliminated.

[0087] Among the sources of environmental errors are ambient light and head gain mismatch. Ambient light is typically the same for both head A and head B of a given pixel circuit 410, and head gain mismatch is not necessarily the same for both head A and head B of a given pixel circuit 410. The environmental errors can be eliminated by an IQ demosaic process, in which the third data block 1030 is converted into a fourth data block 1040 and a fifth data block 1050. The fourth data block 1040 includes data corresponding to the quantity I' in all columns, and the fifth data block 1050 includes data corresponding to the quantity Q' in all columns.

[0088] The IQ demosaic process can be represented by the following expression (1):

[0089] [Mathematical formula 1]

[0090]

[0091] Q, I, Qerror, and Ierror can be given by the following expressions (2)-(4) respectively:

[0092] [Mathematical formula 2]

[0093] Q = xα - yβ (2)

[0094] [Mathematical formula 3]

[0095] I = mα - nβ (3)

[0096] [Mathematical formula 4]

[0097] Q error = I error = γα - γβ (4)

[0098] Above, the quantities x, y, m, and n correspond to active light (e.g., by Figures 1A to 1BThe amount of light emitted by the light generator 111 shown; the quantity γ corresponds to the amount of ambient light; and α and β correspond to the head gain mismatch. By using IQ demosaicing based on the first data block 1010 and the second data block 1020, expression (1) becomes the following expression (5):

[0099] [Mathematical formula 5]

[0100]

[0101] Thereby, the environmental error is eliminated. The quantities I and Q can be obtained in various modes, such as Figures 11A - 14B shown.

[0102] Figures 11A to 11B respectively show the normal IQ mdm modes of the Nth frame and the (N + 1)th frame. In Figure 11A the Nth frame is shown. As shown, the first switch of each of the first switch circuit 431 and the second switch circuit 432 is closed, while the second switch and the third switch of each of the first switch circuit 431 and the second switch circuit 432 are open. The pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at 0 phase, the head B of the corresponding pixel circuit 410 in the left column operates at 180 phase, the head A of the corresponding pixel circuit 410 in the right column operates at 90 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at 270 phase.

[0103] In Figure 11B the (N + 1)th frame is shown. As shown, the states of the first switch circuit 431 and the second switch circuit 432 are the same as those in Figure 11A however, the phases of the pixel circuits 410 are modified. Thereby, the pixel circuits 410 in the bottom four rows are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at 180 phase, the head B of the corresponding pixel circuit 410 in the left column operates at 0 phase, the head A of the corresponding pixel circuit 410 in the right column operates at 270 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at 90 phase.

[0104] In the above manner, the quantities I and Q can be obtained in the Nth frame, and the quantities -I and -Q can be obtained in the (N + 1)th frame. Then, these quantities can be subjected to Figure 10 the IQ mosaic / demosaic processing shown in

[0105] Figures 12A to 12B respectively show the skip 1IQ mdm modes of the Nth frame and the (N + 1)th frame. In Figure 12AIn [the figure], the Nth frame is shown. As shown, the first switch of each of the first switch circuit 431 and the second switch circuit 432 is closed, while the second switch and the third switch of each of the first switch circuit 431 and the second switch circuit 432 are open. The pixel circuits 410 in the first row, the third row, the fifth row, and the seventh row are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at 0 phase, the head B of the corresponding pixel circuit 410 in the left column operates at 180 phase, the head A of the corresponding pixel circuit 410 in the right column operates at 90 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at 270 phase. The pixel circuits 410 in the second row, the fourth row, the sixth row, and the eighth row are skipped.

[0106] In Figure 12B [the figure], the (N + 1)th frame is shown. As shown, the states of the first switch circuit 431 and the second switch circuit 432 are the same as those in Figure 12A [the figure]; however, the phases of the pixel circuits 410 are modified. Thus, the pixel circuits 410 in the first row, the third row, the fifth row, and the seventh row are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at 180 phase, the head B of the corresponding pixel circuit 410 in the left column operates at 0 phase, the head A of the corresponding pixel circuit 410 in the right column operates at 270 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at 90 phase. The pixel circuits 410 in the second, fourth, sixth, and eighth rows are skipped again.

[0107] In the above manner, the quantities I and Q can be obtained in the Nth frame, and the quantities -I and -Q can be obtained in the (N + 1)th frame. Then, these quantities can be subjected to the Figure 10 IQ mosaic / demosaic processing shown in

[0108] Figures 13A to 13B The 2×2 pixel merging IQ mdm modes of the Nth frame and the (N + 1)th frame are shown respectively. In Figure 13A [the figure], the Nth frame is shown. The first switch and the second switch of each of the first switch circuit 431 and the second switch circuit 432 are closed, while the third switch of each of the first switch circuit 431 and the second switch circuit 432 is open. The pixel circuits 410 in the row pairs are driven in four consecutive horizontal periods 1H to 4H, such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at 0 phase, the head B of the corresponding pixel circuit 410 in the left column operates at 180 phase, the head A of the corresponding pixel circuit 410 in the right column operates at 90 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at 270 phase.

[0109] In the first horizontal period 1H, the bottom two rows of pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above-mentioned phases. During the first horizontal period 1H, the signals of the first and second rows for the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals of the first and second rows for the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0110] In the second horizontal period 2H, the third and fourth rows of pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above-mentioned phases. During the second horizontal period 2H, the signals of the third and fourth rows for the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals of the third and fourth rows for the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column. The third horizontal period 3H and the fourth horizontal period 4H follow similarly.

[0111] In Figure 13B , the (N + 1)-th frame is shown. The first switch and the second switch of each of the first switch circuit 431 and the second switch circuit 432 are closed, while the third switch of each of the first switch circuit 431 and the second switch circuit 432 is open. The pixel circuits 410 in the row pairs are driven in four consecutive horizontal periods 1H to 4H such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates at the 180 phase, the head B of the corresponding pixel circuit 410 in the left column operates at the 0 phase, the head A of the corresponding pixel circuit 410 in the right column operates at the 270 phase, and the head B of the corresponding pixel circuit 410 in the right column operates at the 90 phase.

[0112] In the first horizontal period 1H, the bottom two rows of pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above-mentioned phases. During the first horizontal period 1H, the signals of the first and second rows for the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals of the first and second rows for the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0113] In the second horizontal period 2H, the third and fourth row pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above phases. During the second horizontal period 2H, the signals for the third and fourth rows of the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the third and fourth rows of the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column. The third horizontal period 3H and the fourth horizontal period 4H follow similarly.

[0114] In the above manner, the quantities I and Q can be obtained in the Nth frame, and the quantities -I and -Q can be obtained in the (N + 1)th frame. Then, these quantities can be subject to Figure 10 the IQ mosaic / demosaic process shown in

[0115] Figures 14A to 14B The 2×2 pixel merging skip 1IQ mdm modes for the Nth frame and the (N + 1)th frame are shown respectively. In Figure 14A shown is the Nth frame. The first switch and the second switch of each of the first switch circuit 431 and the second switch circuit 432 are closed, while the third switch of each of the first switch circuit 431 and the second switch circuit 432 is open. The pixel circuits 410 in every other pair of rows are driven in two consecutive horizontal periods 1H and 2H such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates in the 0 phase, the head B of the corresponding pixel circuit 410 in the left column operates in the 180 phase, the head A of the corresponding pixel circuit 410 in the right column operates in the 90 phase, and the head B of the corresponding pixel circuit 410 in the right column operates in the 270 phase.

[0116] In the first horizontal period 1H, the bottom two row pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above phases. During the first horizontal period 1H, the signals for the first and second rows of the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the first and second rows of the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0117] In the second horizontal period 2H, the fifth and sixth row pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above phases. During the second horizontal period 2H, the signals for the fifth and sixth rows of the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the third and fourth rows of the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column. The third horizontal period 3H and the fourth horizontal period 4H follow similarly. The third, fourth, seventh, and eighth row pixel circuits 410 are skipped.

[0118] In Figure 14B , the (N + 1)-th frame is shown. The first switch and the second switch of each of the first switch circuit 431 and the second switch circuit 432 are closed, while the third switch of each of the first switch circuit 431 and the second switch circuit 432 is open. The pixel circuits 410 in every other pair of rows are driven in two consecutive horizontal periods 1H and 2H such that in each horizontal period, the head A of the corresponding pixel circuit 410 in the left column operates in the 180 phase, the head B of the corresponding pixel circuit 410 in the left column operates in the 0 phase, the head A of the corresponding pixel circuit 410 in the right column operates in the 270 phase, and the head B of the corresponding pixel circuit 410 in the right column operates in the 90 phase.

[0119] In the first horizontal period 1H, the bottom two row pixel circuits 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above phases. During the first horizontal period 1H, the signals for the first and second rows of the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the first and second rows of the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column.

[0120] In the second horizontal period 2H, the fifth and sixth rows of the driving pixel circuit 410 are driven such that the heads A and B of the corresponding pixel circuits 410 operate in the above phases. During the second horizontal period 2H, the signals for the fifth and sixth rows of the pixel circuits 410 in the left column are supplied to the comparator 450 for the left column through the first switch circuit 431 and the second switch circuit 432 for the left column, while the signals for the third and fourth rows of the pixel circuits 410 in the right column are supplied to the comparator 450 for the right column through the first switch circuit 431 and the second switch circuit 432 for the right column. The third horizontal period 3H and the fourth horizontal period 4H follow similarly. The third, fourth, seventh, and eighth rows of the pixel circuit 410 are skipped.

[0121] In the above manner, the quantities I and Q can be obtained in the Nth frame, and the quantities -I and -Q can be obtained in the (N + 1)th frame. Then, the quantities can be subject to Figure 10 the IQ mosaic / demosaic processing shown in

[0122] Operation method

[0123] An imaging system, such as Figures 1A - 1B the TOF imaging system 101a or the TOF imaging system 101b shown, can be operated to implement any of the above readout modes and thereby provide object detection, depth map generation, face / gesture recognition, imaging, or a combination of the above.

[0124] Figure 15 An exemplary imaging method according to the present disclosure is shown. The imaging method can be implemented by the TOF imaging system 101a or the TOF imaging system 10b. At 1501, a proximity mode selection is made. The selection can be made by a local user, for example, by an operation on a button or a touch screen of a device implementing the TOF imaging system 101a or the TOF imaging system 101b. The selection can also be made by a controller of a device implementing the TOF imaging system 101a or the TOF imaging system 101b, for example, by a remote user request or an automatic or pre-programmed operation. In the proximity mode, at 1502, a low power mode (LPM) can be selected again by the local user and / or the controller of the device. The low power mode can be any one of the above sparsification mode, merging mode, or hybrid mode. At 1503, an object detection determination is made. If no object is detected, the exemplary method can be re-initialized or restarted.

[0125] If an object is detected, the depth measurement mode is selected at 1504. As described above, this selection can be made by a local user and / or the controller of the device. In the depth measurement mode, at 1505, the readout mode is selected by the local user and / or the controller of the device. The readout mode can be any one of the normal mode, pixel sparsification mode, merging mode, IQ mosaic mode, mdm mode, or hybrid mode described above. At 1506, the device generates a depth map. At 1507, the device performs a face recognition operation and / or a gesture recognition operation. In some aspects of the present invention, the exemplary imaging method may only generate a depth map (and not perform the recognition operation) or may only perform the recognition operation (and not generate a full depth map).

[0126] In this way, in 1501 to 1503, the device determines whether an object is present, and if so, in 1504 to 1506 / 1507, the device may generate a depth map and / or perform a recognition operation.

[0127] Figure 16 Another exemplary imaging method according to the present invention is shown. The imaging method can be implemented by a TOF imaging system 10b that incorporates an RGB sensor in addition to the TOF sensor. At 1601, a proximity mode selection is made. This selection can be made by a local user, for example, by operating a button or touch screen on the device implementing the TOF imaging system 101b. This selection can also be made by the controller of the device implementing the TOF imaging system 101b, for example, by a remote user request or an automatic or pre-programmed operation. In the proximity mode, at 1602, the low power mode (LPM) can be selected again by the local user and / or the controller of the device. The low power mode can be any one of the sparsification mode, merging mode, or hybrid mode described above. At 1603, an object detection determination is made. If no object is detected, the exemplary method can be re-initialized or restarted.

[0128] If an object is detected, then at 1604, an RGB camera, such as RGB image sensor 115, may be turned on. The power-on operation may be performed by a local user and / or may occur automatically by a controller of the device. Once the RGB camera is turned on, at 1605, signals from the RGB camera are used to perform a preliminary face recognition operation. Thereafter, at 1606, a readout mode is selected by the local user and / or the device's controller. The readout mode may be any one of the normal mode, pixel sparsification mode, binning mode, IQ mosaic mode, mdm mode, or hybrid mode described above. At 1607, the device generates a depth map. At 1608, the device performs a face recognition operation. The face recognition operation may utilize inputs from the RGB camera (such as the result of the preliminary face recognition operation) and the TOF camera (such as the depth map). Additionally or alternatively, gesture recognition may be performed. In some aspects of the present invention, an exemplary imaging method may produce only a depth map (and not perform a recognition operation) or may only perform a recognition operation (and not produce a full depth map).

[0129] In this manner, at 1601 to 1506, the device determines whether an object is present, and if so, at 1604 to 1607 / 1608, the device may generate a depth map and / or perform a recognition operation.

[0130] Conclusion

[0131] Regarding the processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. have been described as occurring according to a particular ordered sequence, such processes may be practiced using the described steps in an order different from the order described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the description of the processes provided herein is for the purpose of illustrating certain embodiments and should in no way be construed as a limitation on the claims.

[0132] Accordingly, it should be understood that the above description is intended to be illustrative and not restrictive. After reading the above description, many embodiments and applications other than the examples provided will be apparent. The scope should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of equivalents to which those claims are entitled. Future developments are expected and anticipated in the technologies discussed herein, and the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that this application is capable of modification and variation.

[0133] Unless expressly indicated to the contrary herein, all terms used in the claims are intended to be given their broadest reasonable construction and their ordinary meaning as understood by one of ordinary skill in the art to which the technology described herein pertains. In particular, unless the claims recite a specific limitation to the contrary, the use of the singular articles “a,” “the,” “said,” etc. shall be construed to recite one or more of the indicated elements.

[0134] The abstract of the disclosure is provided to enable the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Further, in the foregoing detailed description, it can be seen that for the purposes of streamlining the disclosure, various features are grouped together in each of the embodiments. This method of the disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as reflected by the following claims, the inventive subject matter lies in less than all of the features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description, with each claim standing on its own as a separately claimed subject matter.

[0135] Those skilled in the art will understand that various modifications, combinations, sub - combinations and alterations may occur depending on design requirements and other factors, so long as they are within the scope of the appended claims or their equivalents.

[0136] [List of reference numerals]

[0137] 101a, 101b TOF imaging systems.

Claims

1. A time-of-flight sensor, comprising: A pixel array including a plurality of pixel circuits arranged in an array, wherein a first column of the array includes: A first pixel circuit including a first photodiode, a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode; and A second pixel circuit including a second photodiode, a third capacitor coupled to the second photodiode, and a fourth capacitor coupled to the second photodiode; A first signal line coupled to the first capacitor; A second signal line coupled to the second capacitor; A third signal line coupled to the third capacitor; A fourth signal line coupled to the fourth capacitor; A first switch circuit; A second switch circuit; A first comparator coupled to the first signal line and the third signal line through the first switch circuit; and A second comparator coupled to the second signal line and the fourth signal line through the second switch circuit.

2. The time-of-flight sensor according to claim 1, wherein A second column of the array includes: A third pixel circuit including a third photodiode, a fifth capacitor coupled to the third photodiode, and a sixth capacitor coupled to the third photodiode; and A fourth pixel circuit including a fourth photodiode, a seventh capacitor coupled to the fourth photodiode, and an eighth capacitor coupled to the fourth photodiode; and The time-of-flight sensor further includes: A fifth signal line connected to the fifth capacitor, A sixth signal line connected to the sixth capacitor; A seventh signal line coupled to the seventh capacitor; and An eighth signal line coupled to the eighth capacitor.

3. The time-of-flight sensor according to claim 2, wherein The first comparator is coupled to the fifth signal line through the first switch circuit; and The second comparator is coupled to the sixth signal line through the second switch circuit.

4. The time-of-flight sensor according to claim 2, further comprising: A third switch circuit; A fourth switch circuit; A third comparator coupled to the fifth signal line, the seventh signal line, and the third signal line through the third switch circuit; And A fourth comparator coupled to the sixth signal line, the eighth signal line, and the second signal line through the fourth switch circuit.

5. The time-of-flight sensor according to claim 1, further comprising a timing circuit configured to control the timing of the first switch circuit and the second switch circuit.

6. The time-of-flight sensor according to claim 5, wherein, The timing circuit is configured to cause the time-of-flight sensor to operate in one of a plurality of readout modes based on the states of the first switch circuit and the second switch circuit.

7. The time-of-flight sensor according to claim 6, wherein, The plurality of readout modes include a pixel sparsification mode in which a predetermined number of rows of the pixel array are skipped.

8. The time-of-flight sensor according to claim 6, wherein, The plurality of readout modes include a pixel merging mode in which groups of a predetermined number of pixel circuits are merged.

9. The time-of-flight sensor according to claim 6, wherein, The plurality of readout modes includes a hybrid mode in which a first predetermined number of rows of the pixel array are skipped and a second predetermined number of groups of non-skipped pixel circuits are combined.

10. The time-of-flight sensor according to claim 6, wherein, The plurality of readout modes includes an IQ mode in which the first pixel circuit is driven in a first phase and a second phase complementary to the first phase within one horizontal period.

11. The time-of-flight sensor according to claim 1, further comprising: A reference signal generator configured to output a reference signal to the first comparator and the second comparator.

12. The time-of-flight sensor according to claim 11, wherein, The reference signal has a periodic ramp waveform.

13. A time-of-flight system, comprising: A light source configured to emit light; And A sensor, comprising: A pixel array including a plurality of pixel circuits arranged in an array, wherein the columns of the array include: A first pixel circuit including a first photodiode, a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode; and A second pixel circuit including a second photodiode, a third capacitor coupled to the second photodiode, and a fourth capacitor coupled to the second photodiode; A first signal line coupled to the first capacitor; A second signal line coupled to the second capacitor; A third signal line coupled to the third capacitor; A fourth signal line coupled to the fourth capacitor; A first switch circuit; A second switch circuit; A first comparator coupled to the first signal line and the third signal line through the first switch circuit; and A second comparator coupled to the second signal line and the fourth signal line through the second switch circuit.

14. The time-of-flight system according to claim 13, wherein, The sensor further includes a timing circuit configured to control the timing of the first switch circuit and the second switch circuit.

15. The time-of-flight system according to claim 14, wherein the timing circuit is configured to cause the sensor to operate in one of a plurality of readout modes based on the states of the first switch circuit and the second switch circuit.

16. The time-of-flight system according to claim 15, wherein, The plurality of readout modes includes at least one of a pixel sparsification mode, a pixel merging mode, a hybrid mode, and an IQ mode. In the pixel sparsification mode, a predetermined number of rows of the pixel array are skipped. In the pixel merging mode, groups of a predetermined number of pixel circuits are combined. In the hybrid mode, a first predetermined number of rows of the pixel array are skipped and a second predetermined number of groups of non-skipped pixel circuits are combined. In the IQ mode, the first pixel circuit is driven in a first phase and a second phase complementary to the first phase within one horizontal period.

17. The time-of-flight system according to claim 13, wherein, The light source is a laser diode or a light-emitting diode.

18. The time-of-flight system according to claim 14, wherein, The light has a wavelength in the infrared range.

19. An imaging system, comprising: A first sensor configured to generate image data, the first sensor including a first pixel array; And A second sensor configured to generate distance data, the second sensor including: A second pixel array including a plurality of pixel circuits arranged in an array, wherein one column of the array includes: A first pixel circuit, comprising a first photodiode, a first capacitor coupled to the first photodiode, and a second capacitor coupled to the first photodiode; and A second pixel circuit, comprising a second photodiode, a third capacitor coupled to the second photodiode, and a fourth capacitor coupled to the second photodiode; A first signal line, coupled to the first capacitor; A second signal line, coupled to the second capacitor; A third signal line, coupled to the third capacitor; A fourth signal line, coupled to the fourth capacitor; A first switch circuit; A second switch circuit; A first comparator, coupled to the first signal line and the third signal line through the first switch circuit; and A second comparator, coupled to the second signal line and the fourth signal line through the second switch circuit.

20. The system according to claim 19, wherein The image data corresponds to an RGB representation of an object; and The distance data corresponds to a depth map of the object.

Images