Global shutter pixel circuit and method for computer vision applications

Abstract

An image sensor device includes: a plurality of pixel units arranged in a pixel array; a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Each of the plurality of pixel units includes a photodiode, a storage diode, and a floating diffusion region. The control circuit is configured to activate the photodiode in a plurality of time windows to sense light reflected from a target due to a plurality of corresponding emission light pulses through a predetermined delay time between each time window and a corresponding emission light pulse. The photodiode can be activated using a plurality of bias voltage pulses or a plurality of global shutter signal pulses.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 598,390, filed December 13, 2017, the entire contents of which are incorporated herein by reference.

[0003] The following conventional U.S. patent applications (including the present patent application) are filed concurrently, and the entire disclosures of the other applications are incorporated by reference into the present application for all purposes:

[0004] U.S. patent application Ser. No. 16 / ____, filed on Dec. 13, 2018, and entitled “GLOBAL SHUTTER PIXELCIRCUIT AND METHOD FOR COMPUTER VISION APPLICATIONS”; and

[0005] U.S. patent application serial number 16 / ____, filed on December 13, 2018, and entitled “DIFFERENTIAL PIXELCIRCUIT AND METHOD OF COMPUTER VISION APPLICATIONS” Technical Field

[0006] Embodiments of the present invention generally relate to image sensor pixel circuits and methods for time-of-flight (ToF) depth measurement systems. Background Art

[0007] Image sensors are used in a variety of applications. Examples include digital cameras, mobile phones, home appliances, endoscopes, and satellite telescopes. Some image sensors are implemented using complementary metal oxide semiconductor (CMOS) technology. In these sensors, the number of MOS transistors is equal to the number of pixels. Transistors are used to transform optical images into electrical signals.

[0008] A variety of methods can be used to drive the pixel circuit of a CMOS image sensor. Examples include a rolling shutter (RS) method and a global shutter (GS) method. In the rolling shutter method, a signal is photoelectrically converted by a photoelectric element in each row of a frame. The signal is transmitted to one or more floating diffusion nodes in each row selected in sequence, and then the image signal of the corresponding pixel is output. In the global shutter method, all signals are photoelectrically converted by all photoelectric elements in a frame. The signal is transmitted to one or more floating diffusion nodes at a time. Then, the image signal of the corresponding pixel in the row selected in sequence is output.

[0009] A time-of-flight (ToF) camera is a range-finding imaging camera system that calculates distance by measuring the flight time of the light signal between the camera and the object for each image point, based on the speed of light. With a time-of-flight camera, the entire scene is captured with each laser or light pulse. Time-of-flight camera products have become very popular as semiconductor devices have become fast enough to support such applications. Direct time-of-flight imaging systems measure the direct flight time required for a single laser pulse to leave the camera and reflect back to the focal plane array. 3D images can capture complete spatial and temporal data, recording a complete 3D scene with a single laser pulse. This allows scene information to be quickly acquired and processed in real time, leading to a variety of applications. These applications include automotive applications, human-machine interfaces and gaming, measurement and machine vision, industrial and surveillance measurement, and robotics, among others.

[0010] However, conventional CMOS pixel sensors tend to have disadvantages such as large size and high power consumption. Therefore, an improved pixel circuit and method is needed for various mobile computer vision applications. Summary of the invention

[0011] In an embodiment of the present invention, ToF depth measurement can be performed by illuminating an object or scene with light pulses using a time window sequence and applying convolution processing to the light signal received at the sensor. An embodiment of the present invention provides an improved pixel circuit and method having fast time-of-flight gating using a single control line for each pixel. The pixel circuit can be implemented based on an improved global shutter CMOS image sensor process flow. In some embodiments, a shutter mechanism based on static gate integration combined with dynamic LDM (lateral draining modulation) can be implemented using a global reset gate or drain voltage modulation. Fast signal transmission is achieved in a pixel circuit using low capacitance and low resistance device structures and interconnects.

[0012] According to some embodiments of the present invention, a method of operating a pixel circuit for time-of-flight (ToF) distance measurement is provided. The pixel circuit includes: a photodiode; a drain region adjacent to the photodiode and coupled to a bias voltage; and a shutter gate disposed between the photodiode and the drain region. The shutter gate is controlled by a global shutter signal to apply the bias voltage to bias the photodiode for light sensing. The pixel circuit also has a storage diode and a floating diffusion region. The storage diode is coupled to the photodiode through a first transfer gate controlled by a first transfer signal. The floating diffusion region is coupled to the storage diode through a second transfer gate controlled by a second transfer signal. The method includes an exposure period and a sampling period. During the exposure period, the method includes activating the first transfer gate using the first transfer signal to couple the photodiode and the storage diode; activating the photodiode in a first plurality of time windows to sense light reflected from a target due to a corresponding plurality of emission light pulses, wherein a delay time between each time window and a corresponding emission light pulse is designated as D1. During the sampling period, the method includes activating the second transfer gate using the second transfer signal to transfer charge from the storage diode to the floating diffusion region. The charge in the floating diffusion region is then sampled to determine a sampling signal S1 representing the charge collected during the exposure period.

[0013] According to some embodiments of the present invention, the method may include two exposure and sampling phases. In the first exposure and sampling phase, the photodiode is exposed in a first plurality of time windows in a first exposure period to sense light reflected from a target due to a corresponding plurality of emission light pulses, wherein the delay time between each time window and the corresponding emission light pulse is designated as D1. In the first sampling period, the charge is sampled to determine a first sampling signal S1. In the second exposure and sampling phase, the photodiode is exposed in a second plurality of time windows in a second exposure period to sense light reflected from a target due to a corresponding plurality of emission light pulses, wherein the delay time between each time window and the corresponding emission light pulse is designated as D2. In the second sampling period, the charge is sampled to determine a second sampling signal S2. The method further includes determining a distance to the target based on the first sampling signal S1 and the second sampling signal S2.

[0014] According to some embodiments of the present invention, an image sensor device includes: a plurality of pixel units arranged in a matrix in a pixel array; a control circuit for controlling an exposure phase and a sampling phase of the image sensor device; and a switch circuit for coupling a pixel power line to a first voltage in the exposure phase and coupling the pixel power line to a second voltage in the sampling phase, wherein the first voltage is higher than the second voltage. Each of the plurality of pixel units includes a photodiode located in a semiconductor substrate. The first end of the photodiode is coupled to a bias voltage through a shutter gate controlled by a global shutter signal. A ground contact couples the second end of the photodiode to electrical ground through an electrical ground conductor. Each pixel unit also has a storage diode located in the semiconductor substrate and coupled to the second end of the photodiode through a first transfer gate controlled by a first transfer signal. The pixel unit also has a floating diffusion region located in the semiconductor substrate and coupled to the storage diode through a second transfer gate controlled by a second transfer signal.

[0015] In some embodiments of the above image sensor device, the control circuit is configured to activate the photodiode in a plurality of time windows to sense light reflected from the target due to the corresponding plurality of emission light pulses by a predetermined delay time between each time window and the corresponding emission light pulse. The photodiode can be activated using a plurality of bias voltages or a plurality of global shutter signal pulses.

[0016] According to some embodiments of the present invention, a four-photodiode pixel unit for differential ToF mode operation includes four photodiodes and four storage diodes. Each storage diode is arranged between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharge from one or both of the first adjacent photodiode and the second adjacent photodiode. As used herein, the term "photocharge" refers to the charge generated when light shines on a photodiode. In some cases, the term "photocharge" is used interchangeably with the term "photoelectron". Each photodiode is arranged between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer the photocharge to one or both of the first adjacent storage diode and the second adjacent storage diode.

[0017] In some embodiments of the above pixel unit, the four photodiodes are arranged in a 2×2 array, wherein each storage diode is disposed between two adjacent photodiodes.

[0018] In some embodiments, the pixel unit further comprises a transfer gate located between each pair of adjacent photodiodes and storage diodes. In some embodiments, the pixel unit further comprises a charge control gate located above each storage diode.

[0019] In some embodiments, the pixel unit further has four floating diffusion regions, each of which is disposed adjacent to a corresponding storage diode. The pixel unit further has a transfer gate located between each pair of adjacent storage diodes and the floating diffusion region. In some embodiments, the pixel unit further has a global shutter control gate associated with each photodiode for draining the charge in the photodiode.

[0020] Some embodiments provide an image sensor device comprising the above-mentioned plurality of pixel units arranged in a pixel array.

[0021] According to some embodiments of the present invention, a method for operating a pixel unit includes exposing the pixel unit during an exposure time window, the pixel unit including four photodiodes and four storage diodes. Each storage diode is arranged between two adjacent photodiodes, and each storage diode is configured to receive photocharge from one or both of the two adjacent photodiodes. Each photodiode is arranged between two adjacent storage diodes, and each photodiode is configured to transfer photocharge to one or both of the two adjacent storage diodes.

[0022] In some embodiments, for the differential ToF mode, the above method may further include transferring the collected photocharges from the first pair of photodiodes to a first storage diode disposed between the first pair of photodiodes during a first period; and transferring the collected photocharges from the second pair of photodiodes to a second storage diode disposed between the second pair of photodiodes. The method includes, during a second period, transferring the collected photocharges from the third pair of photodiodes to a third storage diode disposed between the third pair of photodiodes; and transferring the collected photocharges from the fourth pair of photodiodes to a fourth storage diode disposed between the fourth pair of photodiodes. In addition, in order to generate a differential signal, the method includes providing the sum of the photocharges from the first storage diode and the second storage diode to a first input terminal of a differential amplifier; and providing the sum of the photocharges from the third storage diode and the fourth storage diode to a second input terminal of the differential amplifier.

[0023] In some embodiments of the above method, the method further includes transferring photocharge from the first storage diode and the second storage diode to a first floating diffusion region, transferring photocharge from the first storage diode and the second storage diode to a second floating diffusion region, transferring photocharge from the first floating diffusion region to a first sample-and-hold capacitor, and transferring photocharge from the second floating diffusion region to a second sample-and-hold capacitor. The method further includes transferring a signal from the first sample-and-hold capacitor and the second sample-and-hold capacitor to the differential amplifier.

[0024] In some embodiments of the above method, the first pair of photodiodes and the second pair of photodiodes have no common photodiodes; and the third pair of photodiodes and the fourth pair of photodiodes have no common photodiodes.

[0025] In some embodiments of the above method, for binning operation, the method includes transferring photocharges collected in a first pair of adjacent photodiodes to a first storage diode; and transferring photocharges collected in a second pair of adjacent photodiodes to a second storage diode. The method also includes sensing photocharges in the first storage diode and the second storage diode to provide two sensing signals for binning.

[0026] In some embodiments of the above method, for binning operation, the method includes transferring photocharges collected in each photodiode to a corresponding adjacent storage diode; and sensing the photocharges in each storage diode to provide sensing signals for four sub-pixels.

[0027] According to some embodiments of the present invention, a pixel unit may include a plurality of photodiodes and a corresponding plurality of storage diodes. Each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and the second adjacent photodiode. Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharges to one or both of the first adjacent storage diode and the second adjacent storage diode.

[0028] In some embodiments, the pixel unit may further include a plurality of corresponding floating diffusion regions, each of which is disposed adjacent to a corresponding storage diode. In some embodiments, the pixel unit may further include a transfer gate located between each pair of adjacent storage diodes and floating diffusion regions.

[0029] According to some embodiments of the present invention, an image sensing device may include a plurality of pixel units arranged in a pixel array, each pixel unit including four photodiodes, four storage diodes and four floating diffusion regions. Each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and the second adjacent photodiode. Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharges to one or both of the first adjacent storage diode and the second adjacent storage diode. Each floating diffusion region is disposed adjacent to a corresponding storage diode. The image sensing device may include: a first summing device for receiving photocharges from the second floating diffusion region and the fourth floating diffusion region; and a second summing device for receiving photocharges from the first floating diffusion region and the third floating diffusion region. The image sensing device may also include: a differential amplifier coupled to the first summing device and the second summing device; and a control circuit for controlling charge transfer in the image sensing device.

[0030] In some embodiments of the above-mentioned image sensing device, for the differential ToF mode, the control circuit is configured to expose the pixel unit to light during the exposure time window, and, during the first period, transfer the collected photocharge from the first pair of photodiodes to the first storage diode disposed between the first pair of photodiodes, and transfer the collected photocharge from the second pair of photodiodes to the second storage diode disposed between the second pair of photodiodes. During the second period, the control circuit is configured to transfer the collected photocharge from the third pair of photodiodes to the third storage diode disposed between the third pair of photodiodes, and transfer the collected photocharge from the fourth pair of photodiodes to the fourth storage diode disposed between the fourth pair of photodiodes. In addition, the control circuit is configured to generate a differential signal by providing the sum of the photocharges from the second storage diode and the fourth storage diode to the first input terminal of the differential amplifier; and providing the sum of the photocharges from the first storage diode and the third storage diode to the second input terminal of the differential amplifier.

[0031] In some embodiments of the above-mentioned image sensing device, for the binning mode, the control circuit is configured to expose the pixel unit to light during an exposure time window, transfer the photocharges collected in a first pair of adjacent photodiodes to a first storage diode, transfer the photocharges collected in a second pair of adjacent photodiodes to a second storage diode, and sense the photocharges in the first storage diode and the second storage diode to provide two sensing signals for binning.

[0032] In some embodiments of the above-mentioned image sensing device, for the full resolution mode, the control circuit is configured to expose the pixel unit to light during an exposure time window; transfer the photocharges collected in each photodiode to the corresponding adjacent storage diode; and sense the photocharges in each storage diode to provide sensing signals for four sub-pixels.

[0033] In some embodiments of the present invention, an image sensing device may include a photodiode array. The image sensing device also includes a first storage diode, which is arranged between a first pair of photodiodes in the array, and the first storage diode is configured to receive photocharges from each photodiode in the first pair of photodiodes. The image sensing device also includes a first floating diffusion region, which is arranged adjacent to the first storage diode, and further, the image sensing device also includes a second storage diode, which is arranged between a second pair of photodiodes in the array, wherein the second storage diode is configured to receive photocharges from each photodiode in the second pair of photodiodes. The image sensing device also includes: a second floating diffusion region, which is arranged adjacent to the second storage diode; and a circuit, which is configured to receive photocharges from the first floating diffusion region and the second floating diffusion region.

[0034] In some embodiments of the image sensing device, the circuit is configured to sense photocharges in the first storage diode and the second storage diode to provide two sensing signals for binning.

[0035] In some embodiments of the image sensing device, the circuit is configured to sum the photo charges in the first storage diode and the second storage diode.

[0036] In some embodiments, the image sensing device may also have a third storage diode disposed between a third pair of photodiodes in the array, and the third storage diode is configured to receive photocharges from each photodiode in the third pair of photodiodes. The image sensing device may also have a third floating diffusion region disposed adjacent to the first storage diode. A fourth storage diode is disposed between a fourth pair of photodiodes in the array, and the fourth storage diode is configured to receive photocharges from each photodiode in the fourth pair of photodiodes. A fourth floating diffusion region is disposed adjacent to the fourth storage diode. The circuit is further configured to receive photocharges from the third floating diffusion region and the fourth floating diffusion region. In some embodiments, the circuit is further configured to: sum the photocharges in the first storage diode and the second storage diode; sum the photocharges in the third storage diode and the fourth storage diode; and compare the sum of the photocharges in the first storage diode and the second storage diode with the sum of the photocharges in the third storage diode and the fourth storage diode. In some embodiments, the first pair of photodiodes and the second pair of photodiodes have no photodiodes in common; and the third pair of photodiodes and the fourth pair of photodiodes have no photodiodes in common.

[0037] According to some embodiments of the present invention, an image sensing device includes a plurality of photodiodes and a plurality of storage diodes. Each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode. The image sensing device also includes a control circuit that electrically couples each photodiode to a first adjacent storage diode and a second adjacent storage diode. The control circuit is configured to alternate between (i) transferring photocharge from each photodiode to the first adjacent storage diode and (ii) transferring photocharge from each photodiode to the second adjacent storage diode.

[0038] The following description together with the accompanying drawings will provide a further understanding of the nature and advantages of the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a diagram showing a time-of-flight (ToF) imaging system for depth measurement according to an embodiment of the present invention;

[0040] Figure 2A and 2B is a diagram illustrating an example of an image sensor pixel array in a time-of-flight (ToF) imaging system with fiber optical feedback for calibration according to an embodiment of the present invention;

[0041] Figure 3is a diagram showing an optical feedback path in a time-of-flight (ToF) imaging system according to an embodiment of the present invention;

[0042] Figure 4A and 4B is a diagram illustrating a time-of-flight (ToF) imaging system with fiber optical feedback for calibration according to an embodiment of the present invention;

[0043] Figure 5 is a timing diagram illustrating a method for time-of-flight (ToF) depth measurement according to an embodiment of the present invention;

[0044] Figure 6 is a diagram showing a relationship between a sensing signal and a light-to-shutter delay time according to an embodiment of the present invention;

[0045] Fig. 7A is a diagram showing a relationship between a sensing signal and a light-to-shutter delay time of the two signals in the case of two shutters according to an embodiment of the present invention;

[0046] Figure 7B is a diagram showing the relationship between the signals simulated in the case of two shutters and the light-to-shutter delay time of the two signals according to an embodiment of the present invention;

[0047] Figure 7C is a diagram showing the relationship between the signal simulated in the case of two shutters and the depth of the two signals according to an embodiment of the present invention;

[0048] Figure 8 is a timing diagram illustrating a method for calibration and depth measurement in a time-of-flight (ToF) imaging system according to an embodiment of the present invention;

[0049] Fig. 9 is another timing diagram illustrating a method for calibration and depth measurement in a time-of-flight (ToF) imaging system according to an embodiment of the present invention;

[0050] Fig.10 is a flow chart illustrating a method for calibration and depth measurement in a time-of-flight (ToF) imaging system according to an embodiment of the present invention;

[0051] Fig.11 shows a schematic diagram of a pixel circuit for a global shutter image sensor and a cross-sectional view of a portion of a pixel including a photodiode according to some embodiments of the present invention;

[0052] Fig.12 is a waveform timing diagram illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention;

[0053] Fig.13 is a waveform timing diagram illustrating another method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention;

[0054] Fig.14 is a flow chart illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention;

[0055] Fig.15 is a flow chart illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention;

[0056] Fig.16 is a schematic diagram showing another example of a pixel unit according to an embodiment of the present invention;

[0057] Fig.17 is a schematic diagram showing another example of a portion of a plurality of pixel units arranged in a matrix in a pixel array according to an embodiment of the present invention;

[0058] Fig.18 is a cross-sectional view showing a pixel unit device structure according to some embodiments of the present invention;

[0059] Figures 19A to 19F is a top view illustrating layout options for various components in a pixel cell according to some embodiments of the present invention;

[0060] Figures 20A to 20E shows simulation results of pixel units according to some embodiments of the present invention;

[0061] Figures 21A to 21C An interconnection layout structure for a pixel array according to some embodiments of the present invention is shown;

[0062] Figures 22A to 22E shows a lens layout structure for a pixel unit array according to some embodiments of the present invention;

[0063] Fig.23 is a block diagram illustrating an image sensing system according to some embodiments of the present invention;

[0064] Fig.24 is a graph showing experimental results of the relationship between the pixel signal and the laser-to-shutter delay time according to some embodiments of the present invention;

[0065] Fig.25 is a waveform timing diagram illustrating a method for operating a pixel circuit for global shutter image sensing according to some embodiments of the present invention;

[0066] Fig.26is a waveform timing diagram illustrating a method for operating a pixel circuit for rolling shutter image sensing according to some embodiments of the present invention;

[0067] Fig. 27 is a simplified top view showing a pixel unit according to some embodiments of the present invention;

[0068] Fig.28 is a simplified top view schematic diagram showing a differential pixel unit according to some embodiments of the present invention;

[0069] Fig.29A is a simplified circuit diagram illustrating a differential pixel unit according to some embodiments of the present invention;

[0070] Fig.29B is a diagram showing a method for Fig.29A A simplified schematic diagram of a circuit supporting intra-pixel differential mode operation of a pixel circuit of FIG.

[0071] Fig. 30A is a diagram showing some embodiments according to the present invention Fig.28 , 29A and a timing diagram of a time-of-flight (ToF) operation of a pixel unit of 29B;

[0072] Fig. 30B is a diagram showing some embodiments according to the present invention Fig.11 A timing diagram of the time-of-flight (ToF) operation of a pixel unit;

[0073] Fig. 30C is a timing diagram illustrating conventional phase-modulated time-of-flight (ToF) operation;

[0074] Fig.31 is a diagram showing some embodiments according to the present invention Fig.28 A potential diagram of the operation of a photodiode in a pixel unit;

[0075] Fig.32A and 32B is a diagram showing some embodiments according to the present invention Fig.11 A potential diagram of the operation of a photodiode in a pixel unit;

[0076] Fig.33 is a flow chart summarizing a method for differential ToF mode operation;

[0077] Fig.34 is a simplified top view schematic diagram illustrating a pixel unit for binning mode operation according to some embodiments of the present invention;

[0078] Fig.35is a simplified top view schematic diagram illustrating a pixel unit for full resolution mode operation according to some embodiments of the present invention;

[0079] Fig.36 is a diagram showing some embodiments according to the present invention Fig.28 a line drawing of a layout of a portion of a pixel cell; and

[0080] Fig.37A , 37B and 37C are diagrams showing how some embodiments of the present invention may be used. Figures 28 to 29B Simplified timing diagram of the different operating modes implemented by the pixel unit. DETAILED DESCRIPTION

[0081] Embodiments of the present invention provide a system and method for implementing ToF depth measurement with calibration to provide high accuracy using optical feedback and fast image processing. A series of depth measurements can be calibrated for each frame in a manner that has minimal impact on sensor performance and power consumption.

[0082] The following description is given with reference to the series of drawings listed above. These drawings are only examples and should not unduly limit the scope of the claims herein. In combination with the various aspects shown and described, those of ordinary skill in the art will recognize other variations, modifications and substitutions.

[0083] Figure 1 is a diagram showing a time-of-flight (ToF) imaging system for depth measurement according to an embodiment of the present invention. Figure 1 As shown, a time-of-flight (ToF) imaging system 100 (also referred to as a ToF digital camera) includes an illuminator 110 for sending light pulses 112 to illuminate a target object 120 to determine a distance to the target object. The illuminator 110 may include a pulsed illumination unit and an optical device for emitting the light pulses 112 toward the target object. In this example, the illuminator 110 is configured to send light to the target object using, for example, a laser source. However, it should be understood that other electromagnetic radiation sources may also be used, such as infrared light, radio frequency EM waves, etc. The imaging system 100 also includes an image sensor 130 having a gated sensor unit including an array of photosensitive pixels for receiving light signals from the light pulses in a field of view (FOV) 132 of a sensor lens. The pixel array includes an active area and a feedback area, as described below in conjunction with Figure 2A and 2BAs explained. The imaging system 100 also has an optical feedback device 140 for directing a portion of the light from the illuminator 110 to a feedback area of ​​the pixel array. The optical feedback device 140 provides a preset reference depth. The preset reference depth can be a fixed ToF length, which can be used to generate a lookup table (LUT) that associates the sensed light with the depth measurement value. In some embodiments, the optical feedback device can fold the direct light from the illumination unit into the field of view (FOV) of the lens in the sensor unit. The imaging system 100 also includes a ToF timing generator 150 for providing light synchronization and shutter synchronization signals to the illuminator and the image sensor.

[0084] exist Figure 1 In the embodiment of the present invention, the ToF imaging system 100 is configured to send light pulses to illuminate the target object 120. The imaging system 100 is also configured to sense light from the optical feedback device 140 in the feedback area of ​​the pixel array using a shutter window sequence, wherein the shutter window sequence includes a delay time representing a depth range. The depth range may include an entire range of distances that can be determined by the imaging system. The imaging system 100 calibrates the time of flight (ToF) depth measurement reference information based on the sensed light in the feedback area of ​​the pixel array. The imaging system 100 is further configured to sense light reflected from the target object in the active area of ​​the photosensitive pixel array, and determine the distance of the target object based on the sensed reflected light and the calibrated ToF measurement reference information.

[0085] Figure 2A is a simplified diagram showing a pixel array that can be used in an imaging system 100 according to an embodiment of the present invention. As shown, the pixel array 200 includes a plurality of pixels 212, and each pixel in the pixel array includes a photosensitive element (e.g., a photodiode) that converts incident light into an electric current. A fast electronic switch is used as a shutter to control the timing of the light sensing operation. A time-of-flight (ToF) camera acquires a depth image by determining the time it takes for light to travel from a light source to an object and to a camera sensor. This can be accomplished by illuminating an object or scene with light pulses using a sequence of time windows, and applying a convolution process to the light signal received at the sensor. Further details are described below. As Figure 2AAs shown, the pixel array 200 includes an active area 210 and a feedback area 220. The active area can be used to determine the distance of the target object, and the feedback area can be used for depth calibration. The pixel array may also include an isolation area 221 that separates the feedback area 220 from the active area 210 to reduce interference. The size of the isolation area can be selected to prevent light from the feedback loop from contaminating the imaging signal collected by the objective lens. For example, in some embodiments, the isolation area may have a width of about 100 μm to 200 μm. In some embodiments, the feedback area 220 may be located in a portion of the pixel array outside the field of view, such as in a corner or a less used area of ​​the pixel array. Therefore, the dedicated feedback area of ​​the sensor does not generate much overhead. The small feedback area may have a limited number of pixels, such as from a single pixel to a 10×10 pixel array, which allows fast sensing and signal processing. In some embodiments, a larger feedback area may be used to provide a better signal-to-noise ratio (SNR). Averaging the pixels in a small array may improve accuracy. In some embodiments, the feedback area and the active area are exposed separately during the calibration phase. The difference between the two can be used to compensate at run time.

[0086] Figure 2B is a simplified diagram showing a pixel array that may be used in imaging system 100 according to another embodiment of the present invention. Figure 2B As shown, the pixel array 250 is similar to Figure 2A Pixel array 200. However, there may be more than one feedback region. Pixel array 250 includes an active region 210 and two or more feedback regions 220. The pixel array may also include an isolation region 221 that separates each feedback region from the active region. The isolation region may reduce interference between the feedback region and the active region. Pixel array 250 may be used in a ToF imaging system having two illumination sources. In some embodiments, the imaging system may include more than two illumination sources and corresponding feedback sensor regions.

[0087] Figure 3 It is shown Figure 1 A simplified schematic diagram of a portion of a time-of-flight (ToF) imaging system 100 is shown. Figure 3 The optical feedback device is shown configured to prevent light from leaking from the optical feedback device 140 to normal pixels in the array. Light inserted in the edge of the FOV can only illuminate specific pixels in the pixel array, and light with different angles cannot enter the optical device of the sensor.

[0088] In some embodiments, the optical feedback arrangement may be configured to fold direct light from the illumination unit into the field of view (FOV) of a lens in the sensor unit. Figure 4A and 4B is a simplified diagram illustrating a time-of-flight (ToF) imaging system 400 with fiber optic feedback for calibration in accordance with an embodiment of the present invention. Figure 4Ais a top view of the imaging system, Figure 4B 4 is a side cross-sectional view of an imaging system. The imaging system 400 includes an irradiation unit 410 and a sensor unit 430 disposed on a printed circuit board (PCB) 401. Figure 4A and 4B As shown, the illumination unit 410 includes a diode laser source 412, a collimating lens 414, and a diffuser 416 located within an illumination housing 418. The sensor unit 430 includes an image sensor 432, a lens 434, and a lens barrel 436 mounted on the image sensor by an adhesive 438. The imaging system 400 also has an optical fiber 420 that provides a feedback path. In this embodiment, the optical fiber 420 collects a certain amount of light from inside the illumination housing (e.g., from parasitic reflections inside) and then directs the light to a corner 442 of a pixel array 440 of the image sensor 432, but outside the lens barrel 436. In some embodiments, an opaque adhesive 438 blocks light from entering the lens barrel. In this example, the corner area 442 of the pixel array serves as a feedback area for the image sensor.

[0089] Figure 5 is a timing diagram illustrating a method for time-of-flight (ToF) depth measurement according to an embodiment of the present invention. Figure 5 In the figure, the horizontal axis is time and the vertical axis is the intensity or size of the light signal. Waveform 1 represents the light pulse reaching the sensor, which can be reflected from the target or provided by the feedback optical device. Waveform 2 represents the shutter window. It can be seen that the width of the light pulse is W light , the width of the shutter window is W shutter In addition, there is a time delay D between the leading edge of the light and the shutter. L->SH As can be seen, the amount of light sensed by the sensor varies with the relative delay of the shutter with respect to the light.

[0090] Figure 6 is a graph showing the relationship between the magnitude of a sensed light signal and the light-to-shutter delay time according to some embodiments of the present invention. Figure 6 The horizontal axis is the light-to-shutter delay D L->SH, the vertical axis is the amount of light sensed by the sensor. The graph is divided into several regions 601 to 605. In region 601, the shutter window is well ahead of the light pulse (on the left), and the shutter is already closed before the light arrives. In other words, the light-to-shutter delay is negative. Therefore, there is no overlap between the shutter and the light. Moving to the right on the horizontal axis, the delay increases. At point 611, the shutter begins to overlap with the light. When passing through region 602, the delay increases further, the overlap between the shutter and the light continues to increase, and more light is sensed, resulting in a rising curve in region 602. At point 612, the entire width of the light begins to overlap with the shutter window. In region 603, the shutter is fully open for the entire duration of the light pulse, and the width of region 603 is determined by the width W of the shutter opening. shutter Subtract the width W of the light pulse light The amount of light received in this region is labeled as the "Shutter Open (ON) Signal". At point 613, the rising edge of the shutter window is aligned with the rising edge of the light pulse, and the delay D L->SH is zero, as shown at point 617. In region 604, the delay D L->SH The shutter window is open after the light pulse has passed. No light is sensed in region 605, and the amount of light sensed in this region is labeled as a "Shutter OFF Signal". Note that in regions 602 and 604, the amount of light collected by the sensor is proportional to the light-to-shutter delay D. L->SH As described below, these regions are used for ToF depth measurement calibration.

[0091] Fig. 7A is a graph showing the relationship between the light signals sensed in the case of two shutters and the light-to-shutter delay time of the two signals according to an embodiment of the present invention. A time-of-flight (ToF) camera acquires a depth image by determining the time required for light to travel from a light source to an object and then reflect back to the camera. This can be accomplished by illuminating the object or scene with light pulses and applying a convolution of a window sequence with varying delay times to the light signals received by the sensor. In some embodiments, multiple sets of calibrated light pulses are sent using a shutter window sequence that includes a delay time representing a range of depths. Each set of light pulses is followed by a readout operation. At each readout, light from an optical feedback device is sensed in a feedback region of the sensor pixel array. The readout data is then analyzed using a convolution process to determine ToF depth data. As described above, in Figure 6 In regions 602 and 604, the amount of light collected at the sensor depends on the light-to-shutter delay D L->SH It can be collected similar to Figure 6 These areas are used for ToF depth measurement calibration. Fig. 7A As shown, two calibration sequences can be performed to reduce the effect of the unknown reflectivity of the target object; the two sequences are denoted as S1 and S2. In one embodiment, the light-to-shutter delay D of the two sequences is L->SH The difference is equal to the width W of the shutter window shutter In this case, the region 604 of the sequence S1 and the region 602 of the sequence S2 can be Fig. 7A The light collected in S1 and S2 represent two parts of the reflected light pulse, respectively, and the ratio S2 / S1 is related to the corresponding depth or distance to the target object. Fig. 7A The area between points A and B in represents the range of depth that can be determined by this ToF imager. The data of the received light can be collected by measuring at multiple points with the delay between A and B in front of the target. Using a convolution process, a lookup table (LUT) can be constructed that relates the ratio S2 / S1 to the depth or distance to the target. The initial lookup table can be constructed during the factory calibration process. In subsequent ToF depth measurements, the ratio S2 / S1 is calculated from the Fig. 7A The two measurements are made with a delay starting at the same time slice in . The ratio S2 / S1 of the sensed light is determined based on the sensed data and the corresponding depth can be determined from a lookup table.

[0092] Figure 7B is a graph showing the relationship between the simulated signal and the light-to-shutter delay time of the two signals in the case of two shutters according to an embodiment of the present invention. The simulation was performed using two shutters in a static test, where a flat target was located 100 cm from the camera and a certain range of light-to-shutter delays were scanned. Fig. 7A , plotting the shutter signal (or the number of photoelectrons collected at the sensor) for the two shutters S1 and S2. In this view, the depth can be negative. Figure 7B On the horizontal axis, the delay is converted to depth by the following equation:

[0093] <Depth> = <Speed ​​of Light> / 2 * (<Electronic Delay> - <Analog Delay Vector>)

[0094] In some embodiments, the width of the light pulse is 5 to 10 nanoseconds and the shutter window width is 5 to 15 nanoseconds. The delay range examined is between 5 and 20 nanoseconds. In some embodiments, the light pulse width can be between 3 nanoseconds and 20 seconds. The shutter width can be in the same range.

[0095] Figure 7Cis a diagram showing the relationship between the signals simulated in the case of two shutters and the depths of the two signals according to an embodiment of the present invention. Figure 7C shows the wall track at different distances (with 1 / distance 2 It can be seen that there is a correlation between the ratio S2 / S1 and the depth.

[0096] Based on test data (e.g. using Figure 5 , Figure 6 as well as Figures 7A to 7C A lookup table (LUT) is constructed during the factory calibration process based on the data obtained by the method described in . In ToF depth measurement, the ratio S2 / S1 is determined based on the sensed data, and the corresponding depth can be determined from the lookup table.

[0097] As described above, time-of-flight depth measurement systems may be susceptible to variations in process and operating conditions such as temperature, voltage, and frame rate. In order to mitigate the effects of variations, embodiments of the present invention provide a system and method for runtime calibration of ToF depth measurements using the above-mentioned optical feedback device. A small number of feedback pixels allows fast sensing and signal processing, and the number of sampling pulses can be greatly reduced by strong feedback illumination (e.g., illumination provided by optical fiber). The illumination and readout processing can be performed in a short time. Therefore, depth calibration can be performed at runtime without affecting the frame rate of the camera. Calibration can be performed in each frame. In addition, the power consumption or overhead of dedicated feedback pixels is small. Isolation is provided between the feedback area and the active area of ​​the pixel array to minimize interference.

[0098] Figure 8 is a timing diagram illustrating a method for performing depth profile calibration between frames of time-of-flight depth measurement according to an embodiment of the present invention. The method includes a stabilization period 810, a calibration period 820, and a measurement period 830. In the stabilization period 810, a thermally stable illumination pulse is emitted, followed by a dummy readout for thermal stability of the sensor. In the calibration period 820, the time-of-flight lookup table (LUT) is calibrated. Here, multiple groups of calibration illumination pulses P-1, P-2, ..., PN are emitted using a shutter window sequence (including a delay time representing a depth range). Each group of light pulses is followed by a readout operation, namely R-1, R-2, ..., RN. At each readout, light from an optical feedback device is sensed in a feedback region of a pixel array of the sensor. The readout data is then analyzed using convolution processing to determine ToF depth data, as described above in conjunction with Figure 5 , Figure 6 as well as Figures 7A to 7C The depth data is then used to calibrate the lookup table.

[0099] The measurement period 830 has two steps 831 and 832. In the first step 831, a first set of light pulses S1 with a first shutter delay D1 is transmitted to illuminate the target. Because the sensor can only collect a small amount of light within the shutter window, a large number (e.g., thousands) of pulses are typically emitted and collected to increase the signal-to-noise ratio. During the "S1 read" period, the light reflected from the target is sensed in the active area of ​​the pixel in the sensor. In the second step 832, a second set of light pulses S2 with a second shutter delay D2 is transmitted to illuminate the target. During the S2 read, the light reflected from the target is sensed in the active area of ​​the pixel in the sensor. Next, using a calibrated lookup table, the sensed data reading ratio S2 / S1 is used to determine the distance of the target object. In some embodiments, S1 and S2 have preset delays selected during the factory calibration process or in the application field.

[0100] Fig. 9 is a timing diagram illustrating that a depth profile calibration may be adapted between frames of a time-of-flight depth measurement in accordance with an embodiment of the present invention. Fig. 9 Similar to Figure 8 , is used to illustrate an example of the length of time each operation takes within a frame of a time-of-flight depth measurement. In this embodiment, the thermal stabilization pulse takes 0.15 milliseconds and the virtual readout for thermal stabilization takes 0.1 milliseconds. Therefore, the length of the stabilization period is about 0.25 milliseconds. In the lookup table (LUT) calibration period 820, 20 steps of calibration light pulses and readouts are used, each step having a different light-to-shutter delay time. In one example, each step includes 30 pulses, each with a pulse width of 150 nanoseconds, followed by a 3 microsecond readout operation. Therefore, the calibration period takes about 0.15 milliseconds. In the measurement period 830, the S1 step may include a 1.5 millisecond light pulse (e.g., 1000 150 nanosecond pulses), followed by a 0.5 millisecond readout. Similarly, the S2 step may include a 2.0 millisecond light pulse, followed by a 0.5 millisecond readout. In this example, the entire operation including stabilization, full-range depth calibration, and ToF depth measurement takes 4.9 milliseconds. The calibration phase takes about 1 / 300 of the total operation time. The optical operation is fast enough to accommodate frame rates of 60 frames per second (fps) or more.

[0101] Embodiments of the present invention provide many advantages over conventional methods. For example, the feedback optical device can provide strong light for calibration. For example, the feedback optical device can include an optical fiber. One or more separate feedback regions in the pixel array are used to sense the feedback light signal. The feedback region is configured in an unused area or a less used area of ​​the pixel array and is much smaller than the active area of ​​the array. For example, if the feedback optical device can provide a strong signal, a few pixels are sufficient for feedback sensing. A small feedback sensing area allows fast sensing and fast processing of the sensed data, thereby allowing fast calibration of the depth range of interest.

[0102] Fig.10 is a simplified flow chart showing a method for ToF depth measurement including full range depth calibration according to an embodiment of the present invention. The method described above may be Fig.10 . As shown, method 1000 includes sending a light pulse to illuminate a target object at step 1010. Next, at step 1020, light provided from an optical feedback device is sensed in a first area of ​​a photosensitive pixel array. Here, the first area is used as a feedback area. The optical feedback device receives a portion of the sent light pulse. The feedback optical device includes a preset reference depth for ToF depth measurement. Light from the optical feedback device is sampled using a shutter window sequence (including a delay time representing a distance range). For ToF depth measurement, the method includes sensing scene data in a second area of ​​the photosensitive pixel array at step 1030, the scene data being light reflected from a target object based on the sent light pulse. The second area is an active area of ​​the pixel array. The method includes correcting time of flight (ToF) depth measurement reference information based on the sensed light in the first area of ​​the pixel array at step 1040. This process is combined with the above Figure 5 , Figure 6 as well as Figures 7A to 7C Detailed description is given. Note that, depending on the embodiment, steps 1030 and 1040 can be performed in any order. For example, after capturing the calibration data (1020) and the scene data (1040), the data calibration can be processed first and then the scene data. Alternatively, the ToF data calibration and scene data processing can be performed simultaneously. Next, the method includes, at step 1050, determining the distance of the target object based on the sensed reflected light and the calibrated ToF measurement reference information.

[0103] In some embodiments, the method can be performed in a digital camera characterized by a preset frame rate. The calibration can be adapted to a single frame period of the camera. In one embodiment, light from an optical feedback device is sampled using a shutter window sequence (including a delay time representing a range of distances). The measured signal is then associated with the distance using a convolution process. More details of the above method can be found in U.S. Patent Application Serial No. 15 / 721,640, filed on September 29, 2017, entitled “RealTime Calibration for Time-of-Flight Depth Measurement,” the entire contents of which are incorporated herein by reference.

[0104] According to some embodiments of the present invention, an image sensing device may include: a plurality of pixel units arranged in a matrix in a pixel array; and a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Figure 2A and 2B An example of a plurality of pixel units arranged in a matrix in a pixel array is shown. The image sensing device can be used to implement the above-mentioned Figure 8 and 9 Methods for time-of-flight (ToF) depth profiles of some embodiments of the present invention are described. Examples of pixel units that can be used for such imaging devices are described below in conjunction with Figures 11 to 13 describe.

[0105] Fig.11 A schematic diagram 1100 showing a pixel circuit for a global shutter image sensor according to some embodiments of the present invention is shown, as well as a cross-sectional view of a portion of a pixel including a photodiode. The pixel circuit 1110 includes a photodiode 1114 (e.g., a pinned photodiode (PPD)), a storage diode (SD) 1118, and a floating diffusion region (FD) 1119. As used herein, the storage diode (SD) is also referred to as a sensing diffusion region (SD). The pixel circuit 1110 also includes transistors and corresponding control signals. The control signal can be provided by a control circuit in the image sensing device. For example, the global shutter transistor 1111 is coupled to receive a global shutter (GS) control signal, the first transfer gate transistor 1112 receives a first transfer signal (TX1), the second transfer gate transistor 1113 receives a second transfer signal (TX2), and the reset transistor 1115 receives a reset signal (RST). The source follower transistor (SF) 1116 and the select transistor 1117 receive a select (SEL) signal.

[0106] The above pixel circuit can be configured for correlated double sampling (CDS) of the charge in the floating diffusion region to reduce readout noise. In correlated double sampling, at the end of each integration period, the reference voltage of the pixel (i.e., the pixel voltage after reset) is removed from the signal voltage of the pixel (i.e., the pixel voltage at the end of integration).

[0107] Fig.11 A cross-sectional view of a pixel cell 1150 in a portion of a pixel circuit 1110 according to some embodiments of the present invention is also shown. As shown, the pixel cell 1150 includes a substrate 1151, a photodiode (PPD) 1114, and a ground contact for coupling the second end of the photodiode to an electrical ground (VSS) through an electrical ground conductor. The pixel cell 1150 also has a drain region 1153 adjacent to the photodiode and coupled to a bias voltage (VAAGS). In addition, the pixel cell 1150 has a shutter gate 1111 disposed between the photodiode PPD 1114 and the drain region 1153. The shutter gate 1111 is controlled by a global shutter signal (GS) to apply a bias voltage to bias the photodiode (e.g., a pinned photodiode or PPD) for light sensing. A storage diode (SD) 1118 is coupled to the photodiode PPD 1114 through a first transfer gate 1112 controlled by a first transfer signal (TX1).

[0108] Examples of photodiodes may include semiconductor diodes having an N-type region formed in a P-type well. A P-type well may be formed in an N-type substrate. A pinned photodiode (PPD) has an additional P+ pinning layer disposed at the surface of the N-type region, which prevents interface depletion and keeps the photodiode electrically stable.

[0109] Circuit block 1130 includes a row current source transistor 1137 biased by a signal Vbias. In addition, the sample-and-hold circuit includes a first transistor 1131 coupled to a first capacitor 1135. The first transistor 1131 receives a first sample-and-hold signal (SH / S) to sample charge during a signal sampling time. A second transistor 1132 is coupled to a second capacitor 1136. The second transistor 1132 receives a second sample-and-hold signal (SH / R) to sample charge during a reset time.

[0110] Fig.12 is a waveform timing diagram illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention. Fig.11 The pixel circuit 110 described above implements the method by a control signal generated by a control circuit in the image sensing device. An example of an image sensing device is Fig.23For example, Fig.12 As shown, the method includes an exposure period and a sampling period. During the exposure period, a global shutter (GS) signal is turned on in a first plurality of time windows 1210 to activate a photodiode to sense light reflected from a target due to a corresponding plurality of emitted light pulses. Figure 8 and 9 As described above, the delay time between each time window and the corresponding emission light pulse is designated as D1. During this period, the first transfer signal (TX1) 1220 is raised to activate Fig.11 The first transfer gate 1112 in FIG. Fig.11 As shown, this action couples the photodiode (PPD) 1114 and the storage diode (SD) 1118.

[0111] In the sampling period, the background signal is sampled first, and then the image signal is sampled. In order to sample the background signal, the reset (RST) signal 1230 and the selection (SEL) signal 1240 are turned on. The two sample-hold control signals SHR1250 and SHS 1260 are also turned on. Therefore, the background signal (also called the reset level signal) is transmitted to the sample-hold capacitors 135 and 136.

[0112] Next, the second transfer signal (TX2) 1270 is turned on to activate Figure 1 The second transfer gate 1113 in the photodiode (PPD) and storage diode (SD) is transferred to the Fig.11 After the charge is transferred to the floating diffusion region (FD), Fig.12 At 1272 in the example, the second transfer signal (TX2) is turned off. When the SEL and SHS signals are turned on, Fig.11 The charge in the floating diffusion region (FD) is sampled at the sample-and-hold capacitors 135 and 136 in the first exposure phase to perform correlated double sampling. At this time, the charges on the capacitors 135 and 136 can be compared to determine a first sampling signal S1 representing the charge collected during the first exposure phase. This sampling operation corresponds to Figure 8 and 9 The S1 reading period is shown.

[0113] exist Fig.12, VDD is the power supply voltage. In some embodiments, the power supply voltage VDD is at a voltage 1280 during the exposure period, which is higher than the voltage 1290 during the sampling period. For example, the power supply voltage VDD during the exposure period may be at 4.5V (1280), and the transfer signals TX1 and TX2 during the exposure period may be at 3.3V (1290). Therefore, the bias voltage VAAGS or VDD is higher than the first transfer signal (TX1) and the second transfer signal (TX2).

[0114] In some embodiments, by providing a first global shutter signal (GS) having a plurality of global shutter pulses for exposing the photodiodes, Fig.11 The global shutter gate 1111 in can be turned on in the first plurality of time windows to activate the photodiode. In this case, the power supply voltage VAAGS (or VDD) is maintained at a voltage (eg, 4.5V), and the first global shutter signal (GS) is used to apply a pulse signal.

[0115] In some alternative embodiments, the global shutter gate can be turned on in the first plurality of time windows to activate the photodiode by providing a power supply voltage VAAGS (or VDD) with a plurality of pulses for exposing the photodiode. In this case, the first global shutter signal (GS) is maintained at a voltage (e.g., 4.5V), and the power supply voltage VAAGS (or VDD) is used to apply the pulse signal. Fig.13 Further shown in .

[0116] Fig.13 is a waveform timing diagram illustrating another method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention. Fig.13 and Fig.12 Similar, but with one difference. Fig.13 In the global shutter gate ( Fig.11 1111 in receives a plurality of pulse signals VAAGS or VDD. In contrast, Fig.12 In the embodiment, the global shutter gate 1111 receives a plurality of pulse signals from the global shutter control signal (GS). In some embodiments, the signal VAAGS or VDD includes a high frequency pulse signal (e.g., in the nanosecond range). In order to use drain modulation with the signal VAAGS or VDD, the signal Fig.11 The low capacitance drain region 153 in the.

[0117] Fig.14 is a flow chart illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention. Fig.14As shown, method 1400 summarizes the use of the above Figure 8 The above combination used in the first phase of the described stabilization period 810, calibration period 820 or measurement period 830 Figures 11 to 13 A method of describing a pixel circuit.

[0118] Method 1400 includes an exposure period and a sampling period. During the exposure period, at 1410, the method includes activating a first transfer gate using a first transfer signal (TX1) to couple a photodiode (PPD) and a storage diode (SD). At 1420, a plurality of emission light pulses are sent to a target. At 1430, the method includes activating the photodiode in a plurality of time windows to sense light reflected from the target due to the corresponding plurality of emission light pulses. The delay time between each time window and the corresponding emission light pulse is designated as D1. The shutter signal pulse (GS or VDD) is aligned so that the pulse is irradiated with a preset first delay D1.

[0119] In the sampling period, at 1440, the method includes activating the second transfer gate using a second transfer signal (TX2) to transfer charge from the storage diode (SD) to the floating diffusion region (FD). At 1450, the charge in the floating diffusion region is sampled to determine a first sampling signal S1 representing the charge collected during the first exposure period.

[0120] Fig.15 is a flow chart illustrating a method of operating a pixel circuit for time-of-flight (ToF) distance measurement according to some embodiments of the present invention. Fig.15 As shown, method 1500 summarizes the use of the above Figure 8 The above combination used in the measurement period 830 described above Figures 11 to 13 The method 1500 includes two exposure and sampling phases. Fig.15 In the first exposure period, at 1510, the method includes activating a first transfer gate using a first transfer signal (TX1) to couple a photodiode (PPD) and a storage diode (SD). At 1520, a plurality of emitted light pulses are sent to a target. At 1530, the method includes activating the photodiode in a plurality of time windows to sense light reflected from the target due to the corresponding plurality of emitted light pulses. The delay time between each time window and the corresponding emitted light pulse is designated as D1. In the first sampling period, at 1540, the method includes activating a second transfer gate using a second transfer signal (TX2) to transfer charge from the storage diode (SD) to a floating diffusion region (FD). The charge in the floating diffusion region is sampled to determine a first sampling signal S1 representing the charge collected during the first exposure period.

[0121] The method also includes, in a second exposure period, at 1550, activating a first transfer gate using a first transfer signal (TX1) to couple a photodiode (PPD) and a storage diode (SD). At 1560, a second plurality of emitted light pulses are sent to a target, and the photodiode is activated in a corresponding plurality of time windows to sense light reflected from the target due to the corresponding second plurality of emitted light pulses. The delay time between each time window and the corresponding emitted light pulse is designated as D2. At 1570, in a second sampling period, the method includes activating a second transfer gate using a second transfer signal (TX2) to transfer charge from the storage diode (SD) to a floating diffusion region (FD). At 1580, the charge in the floating diffusion region is sampled to determine a second sampling signal S2 representing the charge collected during the second exposure period. At 1590, the method includes determining a distance to the target based on the first sampling signal S1 and the second sampling signal S2.

[0122] Combined with the above Figure 8 and 9 More details about the method are described in detail. For example, the distance to the target can be determined based on the ToF method using a lookup table based on the first sampled signal S1 and the second sampled signal S2. The method can be based on the above combined Figure 8 and 9 Describe the convolution method.

[0123] In some embodiments of the present invention, an image sensor device is provided for performing ToF distance measurement. The image sensing device may include a plurality of pixel units arranged in a matrix in a pixel array, and a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Figure 2A and 2B An example of a plurality of pixel cells arranged in a matrix in a pixel array is shown. The control circuit may include, for example, Figure 1 , Figure 3 as well as Figure 4A and 4B The pulse illumination unit 110, the sensor unit 130 and the ToF timing generator 150 are shown. An example of one of the plurality of pixel units is shown in FIG. Fig.11 Shown in.

[0124] Fig.16 2 is a schematic diagram showing another example of a pixel unit according to an embodiment of the present invention. Fig.16As shown, the pixel unit 1600 includes a photodiode (PPD) located in a semiconductor substrate (not shown), the first end of which is coupled to a bias voltage (VAAGS) through a shutter gate (SGT) and a drain region (Drain) controlled by a global shutter signal (GS). The ground contact (G1) couples the second end of the photodiode to an electrical ground (VSS) through an electrical ground conductor. The pixel unit 1600 also has a storage diode (SD) located in the semiconductor substrate, which is coupled to the photodiode through a first transfer gate (TG1) controlled by a first transfer signal (TX1). The floating diffusion region (FD) in the semiconductor substrate is coupled to the storage diode (SD) through a second transfer gate (TG2) controlled by a second transfer signal (TX2). The switch circuit S1 is configured to couple the pixel power line to a first voltage (V1) during an exposure phase and to couple the pixel power line to a second voltage (V2) during a sampling phase. In this case, the first voltage (V1) can be higher than the second voltage (V2). The pixel unit 1600 also has a source follower (SF) and a selection transistor (SEL) for coupling to a bit line coupled to the sensing circuit. Fig.11 A portion of a sensing circuit is described.

[0125] In some embodiments of the image sensing device, the control circuit is configured to activate the photodiode in multiple time windows to sense the light reflected from the target due to the corresponding multiple emission light pulses with a predetermined delay time between each time window and the corresponding emission light pulse. In some embodiments, the photodiode is activated in multiple time windows using multiple global shutter signal pulses (GS) to sense the reflected light. In other embodiments, the photodiode is activated in multiple time windows using multiple bias voltage pulses (VGSAA) to sense the reflected light. In this case, the photodiode is coupled to the bias voltage (VAAGS) through the drain region in the pixel unit.

[0126] Fig.17 is a schematic diagram showing another example of a portion of a plurality of pixel units arranged in a matrix in a pixel array according to an embodiment of the present invention. Fig.17 Two pixel units are shown: unit 1 and unit 2. Each of unit 1 and unit 2 is similar to the above combination Fig.16 Pixel unit 1600 is described. Fig.17 The dual bit line configuration in odd and even pixels is shown. It can be seen that bit line B1 is associated with cell 1 and bit line B2 is associated with cell 2. The controller can achieve fast readout through the dual bit line configuration of odd / even pixels, as well as the dedicated design of the analog readout mechanism and the fast interface circuit for collecting the image.

[0127] In an embodiment of the present invention, ToF measurement can be achieved using only a single control line for each pixel. In some embodiments, a convolution method for ToF distance measurement is described. The ToF method requires light pulses and shutter windows with different relative delays. For VR and AR applications, the distance of interest may be relatively short, such as room size, in the range of meters rather than kilometers. Therefore, the method requires very narrow light pulses and shutter windows. In order to sample enough light, multiple (e.g., thousands) of emitted light pulses and narrow shutter windows are required in each exposure cycle. Therefore, fast pixel units and circuits are provided. In some embodiments, the pixel circuit can operate in the nanosecond range. In contrast, conventional pixel circuits typically operate in the microsecond range.

[0128] In some embodiments of the present invention, fast time-of-flight (ToF) gating is achieved by using a single control line for each pixel, where the single control line is isolated from other control lines to maintain low capacitance. The single control line can be used to control the above-mentioned global shutter (GS) gate. The global shutter (GS) gate can be controlled by a global shutter signal (GS) or a bias voltage (VAAGS). In contrast, traditional ToF pixel technology can include one of the following two situations: 1) switching two signals of the pixel at the same time, which requires good signal matching; or 2) switching the substrate signal, which requires implementation process complexity.

[0129] In order to achieve fast switching of pixel units, as mentioned above, Fig.11 As described above, the photodiode (PPD), the global gate (GS), and the first transfer gate (TX1) are integrated together. The pixel unit and pixel circuit described herein can be used for ToF imaging, as well as global shutter or rolling shutter imaging with only GS modulation. In some embodiments, a dedicated VSS contact (such as Fig.16 and 17 The resistance between PPD and VSS is reduced by transferring the global lines to a separate metal layer (shown as the dotted line in FIG). The parasitic capacitance between the GS line and other global lines can be reduced. The photodiode is configured to reduce the PPD Vpin (and FWC) to improve the transient time of the charge carriers. The width and length dimensions of the global shutter line are increased to improve the shutter efficiency. In addition, as described above, during the ToF exposure period, a high voltage is used to improve the shutter efficiency. More details of the device structure and layout are described below.

[0130] Fig.18 A cross-sectional view showing a pixel unit device structure according to some embodiments of the present invention is shown. Fig.18In the figure, unit A is a cross-sectional view showing a pixel unit according to some embodiments of the present invention, and unit B is a cross-sectional view showing a traditional pixel unit. It can be seen that unit A has a photodiode (PPD) with a junction depth of about 8μm, while the junction depth of unit B is about 2μm. One way to form a deep junction is to reduce the doping level of the substrate. The deeper junction depth in unit A can improve the quantum efficiency (QE) light sensing in the pixel unit. According to some embodiments, the thickness of the epitaxial layer in which the photodiode is prepared can be in the range of 6 to 9 microns, and the doping density in the epitaxial layer can be between 2E13 and 5E13.

[0131] Figures 19A to 19F is a top view showing layout options for various components in a pixel cell according to some embodiments of the present invention. Figures 19A to 19F In each of the , the pixel unit is shown as having a photodiode (PPD), a transfer gate (TG) and one or two global shutter gates (GS). The dashed line in each figure is the charge watershed, which will be referred to as Figures 20A to 20E The device simulation results shown further illustrate this charge watershed.

[0132] Fig.19A A pixel cell having one global shutter (GS) gate disposed on one side of a photodiode (PPD) and a transfer gate (TG) disposed on an adjacent side of the global shutter (GS) gate is shown. Fig.19B A pixel cell having one global shutter (GS) gate disposed on one side of a photodiode (PPD) and a transfer gate (TG) disposed on an adjacent side of the global shutter (GS) gate is shown. In addition, the transfer gate (TG) is offset relative to the center of the side of the photodiode.

[0133] Fig.19C A pixel cell having one global shutter (GS) gate disposed on one side of a photodiode (PPD) and a transfer gate (TG) disposed on an opposite side of the global shutter (GS) gate is shown. Fig.19D A pixel cell having one global shutter (GS) gate disposed on one side of a photodiode (PPD) and a transfer gate (TG) disposed on the opposite side of the global shutter (GS) gate is shown. In addition, the transfer gate (TG) is offset relative to the center of the side of the photodiode.

[0134] Fig.19E A pixel cell having two global shutter (GS) gates disposed on two adjacent sides of a photodiode (PPD) and a transfer gate (TG) disposed on the other side of the global shutter (GS) gates is shown. Fig.19FA pixel cell having two global shutter (GS) gates disposed on adjacent sides of a photodiode (PPD) and a transfer gate (TG) disposed on the other side of the global shutter (GS) gates is shown. In addition, the transfer gate (TG) is offset relative to the center of the side of the photodiode.

[0135] In some embodiments, Fig.19B The pixel cell shown provides the best pixel device performance. Light integration during the GS ON state may contaminate the GS signal. Therefore, it is desirable to minimize exposure to the TX1 side of the imaging system.

[0136] Figures 20A to 20E Simulation results of pixel units according to some embodiments of the present invention are shown. Fig. 20A A perspective view of a pixel cell is shown showing a photodiode (PPD) with a global shutter gate (GS) and a transfer gate (TX) disposed on adjacent sides of the PPD. Fig. 20A Also shown are the equipotential electrostatic potentials inside the PPD of a pixel cell when integration is on, shutter is open; and when integration is off, shutter is in the closed position. When the shutter is open and integration is on, the entire PPD area is integrated. When the shutter is closed and integration is off, only a small portion of the PPD area is integrated. The dashed line marks the charge watershed. Fig. 20B Shows Fig. 20A Additional simulation results for the device shown.

[0137] Fig. 20C A perspective view of a pixel cell is shown showing a photodiode (PPD) with a global shutter gate (GS) and a transfer gate (TX) disposed on opposite sides of the PPD. Fig. 20C Also shown are the equipotential electrostatic potentials inside the PPD of the pixel cell in the shutter open state and with the shutter in the closed position. The charge watershed is marked by the dashed line. Fig.20D Shows Fig. 20C Additional simulation results for the device shown.

[0138] Fig.20E The simulated equipotential electrostatic potential distribution in a pixel unit having a photodiode (PPD), a transfer gate (TX), and two global shutter gates (GS) (which may be referred to as a double GS) is shown.

[0139] Figures 21A to 21C An interconnection layout structure for a pixel array according to some embodiments of the present invention is shown. Fig.21AA pinned photodiode (PPD) with various control signal lines and power supply lines implemented on different metal layers is shown. For example, the control signal lines for RST, SEL, TX1, and TX2 are implemented in the first metal layer M1. The voltage sources VDDA and VSSA and the output signal line PixelOUT are implemented in the second metal layer M2. The global shutter signal (GS) line is implemented in the third metal layer M3. Note that the global shutter signal (GS) line is a wide metal line in M3, which can reduce the line resistance and capacitance with adjacent lines. Fig.21A Various cross-sectional cut lines AA', BB', CC' and DD' are also identified, which are further exemplified below.

[0140] Fig. 21B Shown along Fig.21A 2. A cross-sectional view of a pixel unit structure along a cutting line. It can be seen from the cross-sectional view along the cutting lines AA' and BB' that the interval between M3 and M1 is large, and the overlap between M2, M3 and M1 is small. Therefore, the capacitance between various metal lines can be reduced.

[0141] Fig. 21C Shown along Fig.21A 2. A cross-sectional view of a pixel unit structure along a cutting line. It can be seen from the cross-sectional view along the cutting lines CC' and DD' that the metal layers M1, M2 and M3 are arranged to be offset from the photodiode (PPD). This layout can provide a large image sensing area in the photodiode.

[0142] Figures 22A to 22E A lens layout structure for a pixel unit array according to some embodiments of the present invention is shown. Figures 22A to 22C A top view of a portion of a pixel unit array is shown, wherein each pixel unit has Figures 21A to 21C The photodiode and metal interconnect structure are shown. Fig.22A An arrangement is shown in which the microlens is located in the center of the pixel cell above the photodiode. Fig. 22B An arrangement is shown in which the microlens is shifted to the right relative to the photodiode in the pixel. Fig. 22C An arrangement is shown in which the microlenses are displaced downward relative to the photodiodes in the pixel. Each lens is appropriately positioned relative to its associated photodiode to direct the primary light arriving through the associated external imaging lens (not shown) to the center of the photodiode, thereby facilitating improved light collection efficiency.

[0143] Fig.22D It is along Figures 22A to 22C A cross-sectional view of a portion of a pixel unit array taken along cutting line EE′ is shown. Fig.22E It is along Figures 22A to 22C A cross-sectional view of a portion of a pixel unit array taken along cutting line FF' is shown. Fig.22D and 22E The advantages of having a wide microlens in the pixel array and light incident into the pixel located at the top of the pixel array are shown for both normal and extreme cases. Fig.22D and 22E As can be seen in Figure 2, since the photodiode is wider, it can collect light from a wide range of light incidence.

[0144] Fig.23 is a block diagram illustrating an image sensing system for evaluating the above method according to some embodiments of the present invention. Fig.23 As shown, the image sensing device or system 2300 may include a plurality of pixel units arranged in a matrix in a pixel array 2310, and a control circuit or controller 2320 having a fast global shutter circuit for controlling the exposure phase and the sampling phase of the image sensor device. The image sensing system 2300 may also include circuits for controlling the timing of illumination, such as a delay generator 2330, a short pulse laser generator 2340, a collimator 2350, and a diffuser 2360. The image sensing system 2300 may also have a processor 2370 and a power supply system 2380. It should be understood that the system is used to evaluate the above method. In actual use, the light emitted by the laser generator 2340 and collimated by the collimator 2350 is reflected by an object in the environment before reaching the pixel array 2310. An imaging lens (not shown) may be disposed in front of the pixel array 2310.

[0145] Combined with the above Figures 11 to 13 Examples of pixel units that can be used in such an image sensing device are described. The image sensing device can be used to implement the above combined Figures 11 to 22E A method of time-of-flight (ToF) depth profiling is described for some embodiments of the present invention.

[0146] In some embodiments, Fig.11 The ToF pixel circuit (which includes circuits for six transistors (6T)) can have a cell pitch of 4.5μm in a 110nm semiconductor manufacturing technology node. In some cases, quantum efficiencies (QE) of about 30% have been achieved. In some embodiments, fast time-of-flight gating enables integration times as short as 5 nanoseconds. Certain embodiments of the present invention include pixels with six transistors and have the following characteristics: the ratio of the cell pitch to the semiconductor node used in manufacturing is less than 50, the quantum efficiency is greater than 25% and the integration time is less than 7 nanoseconds. Different array voltages can be used during global reset and during pixel array readout for efficient transfer / drain conversion. Ambient light is integrated only for the short time (typically ~0.5 milliseconds) that the sensor is turned on for integration. The amount of ambient light for operation depends on the active illumination flux during the integration time.

[0147] Embodiments of the present invention can provide fast time-of-flight gating to achieve short integration times as short as 5 nanoseconds. High quantum efficiency and large optical aperture across the entire pixel array can save infrared illumination power consumption. Image devices and systems have small pixels and low system power, and also have passive and active illumination and ToF, so they can support mobile computer vision systems such as virtual reality, augmented reality, etc. Wide photodiodes and high pixel array resolution are configured to support VGA or higher computer vision applications. There is low noise in global shutter operation and rolling shutter operation modes. Ambient light integration is minimized to enable outdoor operation (e.g., in direct sunlight).

[0148] For example, the capacitance and timing calculations are performed for a 4.5μm VGA array (640×480) on a 110nm process node. The pixel capacitance (including gate + parasitic interconnect) is C pixel =6fF. The resistance of aluminum interconnect is The number of pixels (half VGA row) is n = 320, 240 (for H or V from both sides). The delay of the RC chain from the side of the array to the center of the array can be calculated as follows:

[0149] Elmore Delay:

[0150] Therefore, nanosecond delay can be achieved in the above pixel circuit.

[0151] Fig.24 is a graph showing experimental results of the relationship between the pixel signal and the laser to shutter delay time according to some embodiments of the present invention. This graph is similar to the above Figure 6 as well as Fig. 7A and 7B Figure 1 shows the rise / fall times of about 2 nanoseconds modulated at 3.3V. The pixel signal contrast ratio is about 20:1. Note that the data is measured using a high inductance off-the-shelf package. It is expected that the performance will be improved by using a low inductance package such as a chip-on-board package.

[0152] The pixel circuit and image sensing device are described above in the context of ToF imaging. However, the pixel circuit and image sensing device described above can also be used for global shutter (GS) imaging and rolling shutter (RS) imaging, as described below with reference to Fig.25 and 26 shown.

[0153] Fig.25 is a waveform timing diagram illustrating a method for operating a pixel circuit for global shutter image sensing according to some embodiments of the present invention. Fig.11The pixel circuit 110 described above implements the method. In the global shutter method, all signals are photoelectrically converted by all photoelectric elements in one frame. The signals are transmitted to one or more floating diffusion nodes at a time. Fig.25 As shown, global shutter image sensing includes an exposure period and a sampling period. In the exposure period, together with the first transfer signal (TX1) 2520, the global shutter (GS) gate is turned on (2510) to activate the photodiode to sense the light reflected from the target due to the emitted light. In the sampling period, the operation is similar to the above combined Fig.12 described.

[0154] Fig.26 is a waveform timing diagram illustrating a method for operating a pixel circuit for rolling shutter image sensing according to some embodiments of the present invention. Fig.11 The method is implemented by the pixel circuit 110 described in the rolling shutter method. In the rolling shutter method, a signal is photoelectrically converted by a photoelectric element in each row of a frame. The signal is transmitted to one or more floating diffusion nodes in each row selected in sequence, and an image signal of the corresponding pixel is output. Fig.26 As shown, rolling shutter image sensing may include an exposure period and a sampling period. During both the exposure period and the sampling period, the global shutter (GS) gate is maintained at the side spill potential (2510), and the first transfer signal (TX1) 2620 is in the on state to activate the photodiode to sense the light reflected from the target due to the emitted light.

[0155] According to some embodiments of the present invention, the replacement pixel unit may include a plurality of photodiodes and a corresponding plurality of storage diodes. For example, in some embodiments, each pixel unit includes four photodiodes, four storage diodes and four floating diffusion regions. In other embodiments, different numbers of these components may be utilized. In some embodiments, each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharge from one or both of the first adjacent photodiode and / or the second adjacent photodiode. In addition, each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharge to one or both of the first adjacent storage diode and / or the second adjacent storage diode. In addition, each floating diffusion region is disposed adjacent to the corresponding storage diode.

[0156] In some embodiments, the pixel unit may include an image sensor device that can perform intra-pixel differential ToF determination. In these embodiments, the light reflected from the target is effectively collected, and most of the ambient light is not collected. The intra-pixel differential mode signal is used to determine ToF, and the received ambient light can be further eliminated. In some embodiments, the pixel unit can be used in a binning mode, in which a plurality of photodiodes can be grouped into bins for collecting light. Alternatively, the pixel unit can also be used in a full-resolution mode, in which each photodiode can operate as a sub-pixel. These and other operating modes are described in further detail below. In some embodiments, the image sensor device may be configured to switch between two or more operating modes described herein. For example, in some embodiments, the image sensor can be switched based on user input, sensor data, a predetermined operating schedule, etc., or the image sensor can be instructed to switch from one operating mode to another operating mode. In some embodiments, the image sensor device may not be configured to switch between two or more operating modes described herein, but may be configured to function according to a single operating mode described herein instead.

[0157] Fig. 27 is a simplified top view showing a pixel unit according to some embodiments of the present invention. Fig. 27 As shown, the pixel unit 2700 includes a photodiode 2710, a storage diode (also called a storage diffusion region) 2720, and a readout circuit 2730. In an application, the pixel unit 2700 can be used to implement the above-mentioned pixel unit and pixel circuit, such as Fig.11 1100 in FIG. For example, Fig. 27 The photodiode 2710 in corresponds to Fig.11 The pinned photodiode (PPD) 1114 in FIG. 27 corresponds to the storage diode 2720. Fig.11 The storage diode (SD) 1118 in FIG. 1114 and the readout circuit 2730 corresponds to Fig.11 The circuit block 1130 in FIG. 1 and a portion of the pixel circuit 1110. The term "storage diode" may be used interchangeably with "sensing diffusion region" and "sensing diode".

[0158] In addition, Fig. 27 In the embodiment, the charge transfer from the photodiode 2710 to the storage diode 2720 is performed by the corresponding Fig.11 The charge transfer from the storage diode 2720 to the readout circuit 2730 is controlled by the first transfer signal TX1 on the first transfer gate 2740 of the first transfer gate 1112 in FIG. Fig.11 The second transfer gate 2750 of the second transfer gate 1113 in is controlled by a second transfer signal TX2.

[0159] Fig.28 is a simplified top view schematic diagram showing a differential pixel unit according to some embodiments of the present invention. Fig.28 As shown, the pixel unit 2800 includes four photodiodes: a first photodiode PPD1, a second photodiode PPD2, a third photodiode PPD3, and a fourth photodiode PPD4. The pixel unit 2800 also includes four storage diodes: a first storage diode SD1, a second storage diode SD2, a third storage diode SD3, and a fourth storage diode SD4.

[0160] The pixel unit 2800 also includes four readout circuits: a first readout circuit RO_1, a second readout circuit RO_2, a third readout circuit RO_3, and a fourth readout circuit RO_4. Each readout circuit includes a floating diffusion region disposed adjacent to a corresponding storage diode. For example, the first readout circuit RO_1 includes a first floating diffusion region FD1 disposed adjacent to a first storage diode SD1, the second readout circuit RO_2 includes a second floating diffusion region FD2 disposed adjacent to a second storage diode SD2, the third readout circuit RO_3 includes a third floating diffusion region FD3 disposed adjacent to a third storage diode SD3, and the fourth readout circuit RO_4 includes a fourth floating diffusion region FD4 disposed adjacent to a fourth storage diode SD4. The floating diffusion region is a charge storage region for holding photocharges to be transferred to the readout circuit.

[0161] also, Fig.28 Two charge summing devices AMP1 and AMP2 are shown, which are configured to integrate the photocharges from the two readout circuits. The charge summing devices can be implemented using sample-and-hold capacitors, summing amplifiers, etc. In these embodiments, the charge summing devices AMP1 and AMP2 are not included in the pixel unit 2800, as shown by their location outside the dashed line representing the pixel unit 2800. The charge summing on PPD1 / PPD4 to PPD2 / PPD3 can be performed outside the pixel array, for example, on a sample-and-hold (S / H) capacitor located in a column parallel circuit, just before it is converted to digital using an ADC (analog-to-digital converter) circuit.

[0162] Similar to Fig. 27 The pixel unit 2700 shown in FIG. Fig.28 In each photodiode in the pixel unit 2800 shown, the transfer of photocharge from the photodiode (PPD) to the storage diode (SD) is controlled by the first transfer gate signal TX1, and the transfer of photocharge from the storage diode (SD) to the floating diffusion region (FD) in the readout circuit is controlled by the second transfer gate controlled by the second transfer signal TX2. Fig.28, the first transfer signal TX1 is divided into four signals having different phases: TX1_1, TX1_2, TX1_3, and TX1_4, and the second transfer signal TX2 is divided into two transfer signals having different phases: TX2_1 and TX2_2.

[0163] In some embodiments, the pixel unit 2800 may further include four charge control gates responsive to the charge control signal TX3, each charge control gate being disposed above the storage diode. Fig.28 In the figure, each charge control gate is shown as a dashed rectangle located above the corresponding storage diodes SD1, SD2, SD3 and SD4, respectively. In some embodiments, the charge control gate is controlled by a common storage diode charge control signal TX3. The charge control signal TX3 maintains the charge control gate at a constant voltage to keep the photocharge in differential ToF mode. TX3 can prevent spillback and reduce the toggled gate capacitance. TX3 can also maintain the shape of the well and the potential slope required for charge storage and transfer. In standard mode and during readout, TX3 is switched simultaneously or in parallel with the transfer gate TX1_N, where N=1, 2, 3 or 4, so they act as a single gate. In some embodiments, there is no implanted doping region between TX3 and the transfer gate TX1_N.

[0164] Fig.29A is a simplified circuit diagram illustrating a differential pixel cell according to some embodiments of the present invention. Fig.29A 2900 is shown for the differential pixel unit (i, j) of the i-th column and j-th row in the pixel array. Fig.29A As shown, the pixel unit 2900 includes a first photodiode PPD1(i, j), a second photodiode PPD2(i, j), a third photodiode PPD3(i, j) and a fourth photodiode PPD4(i, j). The pixel unit 2900 also includes a first storage diode SD1(i, j), a second storage diode SD2(i, j), a third storage diode SD3(i, j) and a fourth storage diode SD4(i, j). The pixel unit 2900 also includes four readout circuits: a first readout circuit RO_1, a second readout circuit RO_2, a third readout circuit RO_3 and a fourth readout circuit RO_4. In an application, the circuit of the pixel unit 2900 can be used to implement one or more components of the pixel unit 2800, as described above with reference to FIG. Fig.28 described.

[0165] exist Fig.29A, the transfer signal TX1_1 controls the transfer of photocharges from the first photodiode PPD1(i,j) to the first storage diode SD1(i,j) through the control gate 2911. The transfer signal TX1_2 controls the transfer of photocharges from the fourth photodiode PPD4(i,j) to the first storage diode SD1(i,j) through the control gate 2912. The charge transfer from the first storage diode SD1(i,j) to the first readout circuit RO_1 is controlled by the transfer signal TX2_2 on the transfer gate 2922.

[0166] Similarly, transfer signal TX1_3 controls the transfer of charge from the first photodiode PPD1(i,j) to the second storage diode SD2(i,j) through control gate 2913. Transfer signal TX1_4 controls the transfer of charge from the fourth photodiode PPD4(i,j) to the second storage diode SD2(i,j) through control gate 2914. The charge transfer from the second storage diode SD1(i,j) to the second readout circuit RO_2 is controlled by transfer signal TX2_1 on transfer gate 2921.

[0167] In addition, Fig.29A , the transfer signal TX1_2 controls the transfer of charge from the third photodiode PPD2(i,j) to the third storage diode SD3(i,j) through the control gate 2932. The transfer signal TX1_1 controls the transfer of charge from the third photodiode PPD3(i,j) to the third storage diode SD3(i,j) through the control gate 2931. The charge transfer from the third storage diode SD3(i,j) to the third readout circuit RO_3 is controlled by the transfer signal TX2_1 on the transfer gate 2941.

[0168] Similarly, transfer signal TX1_4 controls the transfer of charge from the second photodiode PPD2(i,j) to the second storage diode SD2(i,j) through control gate 2934. Transfer signal TX1_3 controls the transfer of charge from the third photodiode PPD3(i,j) to the fourth storage diode SD4(i,j) through control gate 2933. The charge transfer from the third storage diode SD3(i,j) to the fourth readout circuit RO_4 is controlled by transfer signal TX2_2 on transfer gate 2942.

[0169] The operation of the readout circuits RO_1, RO_2, RO_3 and RO_4 is similar to the above combined Fig.11 Therefore, only a simplified circuit is shown in each of the readout circuits RO_1, RO_2, RO_3, and RO_4, including the floating diffusion region FD1. <j>、FD2 <j><h2 style=";text-align:left;direction:ltr">FD3<h2 style=";text-align:left;direction:ltr"> <j>and FD4 <j>; Four for receiving reset signal RST <j>The control gate of the four source follower transistors SF1 <j><h2 style=";text-align:left;direction:ltr">SF2<h2 style=";text-align:left;direction:ltr"> <j><h2 style=";text-align:left;direction:ltr">SF3<h2 style=";text-align:left;direction:ltr"> <j>and SF4 <j>, and another four for receiving the selection signal SEL <j>The floating diffusion regions in the readout circuits RO_1, RO_2, RO_3, and RO_4 are similar to the reference Fig.28 A floating diffusion region for receiving charge from a memory device is described. Fig.29B The source follower (SF) and reset gate (RST) in the Fig.11 The corresponding elements in the same way have similar functions.

[0170] The pixel unit 2900 also includes a global shutter gate for coupling each photodiode to a power supply VDD, and a global shutter signal (GS) for controlling the global shutter gate. In FIG. 29 , in order to simplify the drawing, only one global shutter gate (2901) is shown as being associated with the photodiode PPD1 (i, j) to represent a global shutter gate that supports the global shutter signal (GS) and the power supply VDD. However, it should be understood that each of the four photodiodes has a global shutter gate associated with each photodiode to enable the corresponding global shutter signal (GS). In some embodiments, the global shutter signal (GS) has two functions. The first function is to provide an overflow drain to avoid bloom, and the second function is to enable a global reset to clear the photodiodes outside the integration sequence.

[0171] In some embodiments, readout circuits RO_1, RO_2, RO_3, and RO_4 may be shared with neighboring pixel cells. Fig.29A Some circuit components and signals associated with adjacent pixel units are shown in FIG. 1 , such as TX2_1(i-1,j), SD3(i-1,j), TX2_2(i,j-1), SD4(i,j-1), TX2_1(i,j+1), SD2(I,j+1), TX2_2(i+1,j), and SD1(i+1,j). These arrangements can facilitate simultaneous or parallel charge transfer starting from SD2 and SD4, and simultaneous or parallel charge transfer starting from SD1 and SD3.

[0172] Fig.29B is a diagram showing a method for Fig.29A A simplified schematic diagram of the supporting circuitry for intra-pixel differential mode operation of the pixel circuit 2900. Fig.29B As shown in the circuit 2950, ​​the summing device AMP1 includes sampling and holding capacitors SH2 and SH4, which are used to respectively Fig.29A Similarly, the summing device AMP2 includes sampling and holding capacitors SH1 and SH3 for respectively obtaining the output signal from Fig.29A The readout circuits RO_1 and RO_3 in the circuit 2950 receive the output signal. The circuit 2950 also includes a differential amplifier 2970 for receiving the signal from the sample-and-hold capacitor to determine a differential signal.

[0173] Fig. 30A , 30B 30C are timing diagrams showing time-of-flight (ToF) operations of three different pixel cells. Fig. 30A is a diagram showing some embodiments according to the present invention Fig.28 , 29A and 29B are timing diagrams for differential pulse time-of-flight (ToF) operation of the pixel unit. Fig. 30A In FIG. 1 , curve 3010 represents light pulses emitted from a light source toward a target for ToF determination. Curve 3011 represents ambient light signals. The ToF operating range is defined by the interval between the shortest distance and the longest distance measurable using a ToF system. Timing curve 3016 represents an exposure time window during which the photodiode is enabled to sense light input. In some embodiments, timing curve 3016 may represent a combination of Figures 11 to 13 Control signal in the side leakage arrangement described. Fig. 30A As shown, the duration of timing curve 3016 (which may be referred to as lateral drain) extends from the time before the start of timing pulse #1 3001 (associated with the shortest distance measured from the target to the pixel unit) to the time after the end of timing pulse #4 3004 (associated with the longest distance measured from the target to the pixel unit). That is, the duration of timing curve 3016 is greater than the duration associated with the ToF operating range. The timing curve 3016 associated with the lateral drain may be used to discharge ambient light received outside the operating window associated with the ToF operating range, thereby reducing photocharge associated with the ambient light and improving the signal-to-noise ratio.

[0174] Timing curves 3012 and 3014 represent timing windows during which charge is transferred from the photodiode to the storage diode. The charge in the storage diode is then transferred to the floating diffusion region in the readout circuit. The four timing pulses are labeled 3001, 3002, 3003, and 3004, respectively. During these timing pulses, photons collected by the photodiode can be directed to one of the one or more summing devices (e.g., AMP1 or AMP2) described more fully below, depending on the arrival time of the photons.

[0175] The duration associated with the ToF operating range extends from the beginning of timing pulse #1 3001 to the end of timing pulse #4 3004. For example, in different ToF frames, the ToF target can move slowly and monotonically from a short distance to a long distance. When the light pulse 3010 is emitted toward the target and reflected back from the target, first, the reflected light is captured by pulse #1, then by pulses #1 and #2, pulse #2, pulse #2 and pulse #3, pulse #3, pulse #3 and pulse #4, and pulse #4. Thereafter, for example, when the target moves outside the ToF operating range, the integration of the reflected light stops.

[0176] refer to Fig.28 , AMP2 (in Fig. 30A 2) combines the charges from SD1 and SD3 during timing pulses 1 and 3, and AMP1 (referred to as Memory 2 in the timing diagram ... Fig. 30A The timing diagram of FIG. 1 is referred to as memory 1) combines the charges from SD2 and SD4 during timing pulse #2 3002 and timing pulse #4 3004. As shown above in Fig.29B As shown in FIG. 1 , the output signals from AMP1 and AMP2 are coupled to a differential amplifier 2970 to produce a differential signal. When the returned light is integrated between timing pulse #1 (in 3012) to timing pulse #4 (in 3014), according to Fig. 30A The differential signal generated by the timing diagram in can vary with respect to the ToF depth / time shift. Therefore, a differential signal based on the light reflected from the target is used to determine the ToF. In addition, the differential signal can eliminate or reduce the contribution from ambient light.

[0177] In differential mode operation, you can refer to Fig.28 , 29A 29B describe charge transfer. In a complementary manner, transfer signals TX1_3 and TX1_4 are switched together, and transfer signals TX1_1 and TX1_2 are switched together. For example, during charge summation in phase 1, Fig. 30A As shown in the timing pulse #1 3001 in FIG. 1 , under the control of the transfer signals TX1_1 and TX1_2, the photocharges from the photodiodes PPD1 and PPD4 are transferred to the storage diode SD1, and the photoelectrons from the photodiodes PPD2 and PPD3 are transferred to the storage diode SD3. The charges in the storage diodes SD1 and SD3 are transferred to the floating diffusion regions in the corresponding readout circuits, and the output signals from the readout circuits RO_1 and RO_3 are then transmitted to the amplifier AMP2, where the charges can be stored in the sample-and-hold capacitor.

[0178] Similarly, during the charge summation in phase 2, as Fig. 30A As shown in the timing pulse #2 3002 in FIG. 1 , under the control of the transfer signals TX1_3 and TX1_4, the photocharges from the photodiodes PPD1 and PPD2 are transferred to the storage diode SD2, and the photocharges from the photodiodes PPD3 and PPD4 are transferred to the storage diode SD4. The charges in the storage diodes SD2 and SD4 are transferred to the floating diffusion regions in the corresponding readout circuits, and the output signals from the readout circuits RO_2 and RO_4 are input to the amplifier AMP1, where the charges can be stored in the sample-and-hold capacitor.

[0179] Similarly, during timing pulse #3 3003, under the control of transfer signals TX1_1 and TX1_2, photocharges from photodiodes PPD1 and PPD4 are transferred to storage diode SD1, and photocharges from photodiodes PPD2 and PPD3 are transferred to storage diode SD3. The charges in storage diodes SD1 and SD3 are transferred to floating diffusion regions in the readout circuit. The output signals from readout circuits RO_1 and RO_3 are then transmitted to amplifier AMP2, where the charges can be stored in the sample-and-hold capacitor.

[0180] During timing pulse #4 3004, under the control of transfer signals TX1_3 and TX1_4, photocharges from photodiodes PPD1 and PPD2 are transferred to storage diode SD2, and photocharges from photodiodes PPD3 and PPD4 are transferred to storage diode SD4. Charges in storage diodes SD2 and SD4 are transferred to floating diffusion regions in the readout circuit. Output signals from readout circuits RO_2 and RO_4 are input to amplifier AMP1, where charges can be stored in sample-and-hold capacitors.

[0181] In some embodiments, transfer signals TX2_1 and TX2_2 are used to transfer photocharges from the storage diode to the floating diffusion region. Fig.28 As shown, under the control of transfer signal TX2_1, the photocharge from storage diode SD2 is transferred to floating diffusion area FD2, and the photocharge from storage diode SD3 is transferred to floating diffusion area FD3. Similarly, under the control of transfer signal TX2_2, the photocharge from storage diode SD1 is transferred to floating diffusion area FD1, and the photocharge from storage diode SD4 is transferred to floating diffusion area FD4. In some embodiments, these charge transfers can be Fig.28 In an alternative embodiment, these charge transfers may occur after timing pulse #4 in Fig.28 It occurs after timing pulse #3 in , and then after timing pulse #4.

[0182] like Fig.29B As shown, the output signals from amplifiers AMP1 and AMP2 are then provided to differential amplifier 2970 where a differential signal is formed. Thus, a differential signal based on the light reflected from the target is used for ToF determination. As described above, since the returned light is integrated between timing pulse #1 (in 3012) and timing pulse #4 (in 3014), according to Fig. 30A The timing diagram in Figure 2 produces a differential signal that can vary with respect to the ToF depth / time shift.

[0183] like Fig. 30A As shown, substantially all light reflected from the target can be collected. In this embodiment, a high-intensity light source is used as the emitter compared to the ambient light signal 3011. Therefore, in some embodiments, a small amount of ambient light is collected compared to the reflected light. Light pulse 3010 is sent to the target and returns. The time range between the start of timing pulse #1 of 3012 and the end of timing pulse #4 of 3014 represents the ToF operating range. Within this range, all reflected light is collected. In contrast, most of the ambient light is outside this time range and is therefore not collected. Another advantage of the intra-pixel differential mode of ToF is that the effect of ambient light can be further reduced or eliminated. After the time range of ToF operation, the photocharge can be drained. In some embodiments, a global shutter gate is disposed between the photodiode and the power supply, and turning on the global shutter gate can drain most of the collected photocharge. For example, 90% of the photocharge can be drained from the photodiode. This specific percentage is representative only, and other percentages are also included in the scope of the present invention. Although the above mainly refers to four timing pulses (e.g., timing pulse #1 3001, timing pulse #2 3002, timing pulse #3 3003, and timing pulse #4 3004), Fig. 30A , however, it should be understood that in some embodiments, one or more of the image sensing devices described herein may employ fewer or more than four timing pulses in each ToF measurement cycle.

[0184] Fig. 30B is a diagram showing some embodiments according to the present invention Fig.11 Timing diagram of the time-of-flight (ToF) operation of a pixel unit. Fig.11 The ToF operation of the pixel unit is combined with the above Figures 5 to 11 A timing diagram is described in which the ToF swings between the storage diode (SD) and the global shutter (GS). This timing diagram can be associated with a drain-only ToF mode of operation. Fig. 30B In FIG. 30, curve 3020 represents light pulses emitted from a light source toward a target to determine ToF. Curve 3021 represents an ambient light signal. Timing curve 3022 represents an exposure time window during which the photodiode is enabled to sense light input for pixel modulation. In this embodiment, a high intensity light source is used as an emitter compared to the ambient light signal 3021. Therefore, in some embodiments, a small amount of ambient light is collected compared to the reflected light. Therefore, in this embodiment, only a few emitted light pulses are utilized, one of which is at Fig. 30B Similar to the above combination Fig. 30A As described above, most of the reflected light is collected and most of the ambient light 3021 is rejected. However, since the output signal is non-differential, further ambient light elimination cannot be achieved. Some energy of the laser light emitted from the light source is leaked away, for example, in some embodiments, light received at the pixel unit at a time outside the exposure time window is leaked away.

[0185] Fig. 30C is a timing diagram showing conventional phase modulated time-of-flight (ToF) operation. Fig. 30C , 3030 represents the laser emitted from the light source, and 3031 represents the ambient light. In addition, 3032 and 3035 represent timing pulses for determining the phase-modulated time of flight (ToF). For each light pulse emitted toward the target, the light reflected from the target is collected in two consecutive timing windows in 3032 and 3035, respectively. ToF information is derived from the two sensed reflected light signals. In this ToF method, no ambient light is rejected and all laser energy is collected. In addition, the differential output can eliminate ambient light.

[0186] Fig.31 is a diagram showing some embodiments according to the present invention Fig.28 Potential diagram of the operation of the photodiode in a pixel unit. Fig.28 The pixel unit is 2800, Fig.31 The potentials across the photodiode PPD4, the storage diodes SD1 and SD4 on both sides of PPD4, the floating diffusion area FD, and the power supply VDD are shown. The transfer signal TX1_2 controls the charge transfer from PPD4 to SD1, and the transfer signal TX1_3 controls the charge transfer from PPD4 to SD4. The transfer signal TX2 controls the charge transfer from the storage diodes SD1 and SD3 to the floating diffusion area FD. The reset signal RST applies the power supply to reset the circuit. As shown above in conjunction with Fig. 30A As described in the timing diagram in , the transfer signals TX1_1 and TX1_3 switch in a complementary manner, that is, when one is turned on, the other is turned off, and vice versa. In other words, the transfer gate TX1 is divided into a plurality of transfer gates, such as TX1_1, TX1_2, TX1_3, and TX1_4, for the modulated and unmodulated parts. Fig.31 , the solid line shows the transfer of photocharge from the photodiode PPD4 to the storage diode SD4 when the transfer signal TX1_3 is turned on. The dotted line shows the transfer of charge from the photodiode PPD4 to the storage diode SD1 when the transfer signal TX1_2 is turned on. The dotted line also shows the transfer of charge from the storage diode SD1 to the floating diffusion area FD under the control of the transfer signal TX2, and the leakage of charge under the control of the reset signal RST. As shown in the figure, TX1_2 and TX1_3 switch in a complementary manner. In other words, when one of TX1_2 and TX1_3 is high, the other is low, and vice versa. This arrangement can prevent charge overflow.

[0187] As mentioned above, Fig.28 Each photodiode in the pixel unit 2800 is coupled to two storage diodes located on both sides of each photodiode. During the first time window, the charge in the photodiode PPD4 is transferred to the storage diode SD4, such as Fig.31 During the second time window, the charge in the photodiode PPD4 is transferred to the storage diode SD1, as shown in the real potential curve in FIG. Fig.31 As shown in the virtual potential curve in A. At this time, the charge is held in SD1 and SD4 respectively to be transferred to the floating diffusion area in the readout circuit. The charge control signal TX3 controls the charge control gate set above the storage diode to prevent the charge from leaking back to the PPD and allows the gate area to be reduced, thereby reducing the capacitance of TX1_N. The reduced capacitance and leakage can achieve faster ToF modulation with higher contrast. Therefore, Fig.28 The pixel cells act as charge storage cells to allow charge to be collected during different timing windows, thereby improving ToF modulation performance. The pixel cells are also configured to prevent charge overflow, reduce gating capacitance (which may affect gating speed and contrast), and improve modulation contrast. In addition, the pixel cells can also achieve differential and non-differential image capture, as shown below in combination Fig.34 and 35 In addition, the above pixel unit is also compatible with the pinned photodiode global shutter pixel design.

[0188] Fig.32A and 32B is a diagram showing some embodiments according to the present invention Fig.11 Potential diagram of the operation of the photodiode in the pixel unit. Fig.11 In a pixel cell, each photodiode (PPD) is coupled to only one storage diode (SD). Fig.31 In , turning on the transfer gate TX1 transfers the photocharge from the photodiode (PPD) to the storage diode (SD). The photocharge is then transferred to the readout circuit. Fig.32A In the embodiment of the present invention, after the exposure period, turning on the global shutter (GS) can drain the charge in the photodiode. Therefore, the photodiode is no longer available to provide more photocharges.

[0189] In some embodiments of the present invention, Figures 28 to 31 The pixel unit shown may be used for ToF distance measurement in an image sensor device.The image sensing device may include a plurality of pixel units arranged in a matrix in a pixel array, and a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Figure 2A and 2B An example of a plurality of pixel cells arranged in a matrix in a pixel array is shown.

[0190] exist Figures 28 to 31 In the pixel unit shown, each pixel unit includes four photodiodes, four storage diodes and four floating diffusion areas. Each storage diode is arranged between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharge from one or both of the first adjacent photodiode and / or the second adjacent photodiode. In addition, each photodiode is arranged between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharge to one or both of the first adjacent storage diode and / or the second adjacent storage diode. In addition, each floating diffusion area is arranged adjacent to the corresponding storage diode.

[0191] In alternative embodiments, the pixel unit is not limited to four photodiodes. Typically, the pixel unit may include a plurality of photodiodes and a corresponding plurality of storage diodes. The methods described herein may be used for pixels having more than four pixel photodiodes. Similarly, in some embodiments, the methods described herein may be used for pixels having less than four photodiodes (e.g., two photodiodes or three photodiodes). In addition, in some embodiments, the pixel unit is not limited to four storage diodes. The methods described herein may be used for pixels having more than or less than four storage diodes. In some embodiments, the pixel unit may include an unequal number of photodiodes and storage diodes. For example, in such an embodiment, the pixel unit may include an odd number of photodiodes (e.g., five photodiodes) and an even number of storage diodes (e.g., four storage diodes) arranged in a checkerboard configuration. In this example, each storage diode may be disposed between more than two photodiodes (e.g., three photodiodes or four photodiodes) and is also configured to receive photocharges from more than two photodiodes. One of ordinary skill in the art will recognize many variations, modifications, and substitutions.

[0192] like Fig.28 As shown in FIG. 29 , the image sensor device further includes a first summing device for receiving photocharges from the second floating diffusion area and the fourth floating diffusion area, and a second summing device for receiving photocharges from the first floating diffusion area and the third floating diffusion area. Fig.29B As shown, the image sensor device may also include a differential amplifier coupled to the first and second summing devices. The image sensor device may also have a control circuit for controlling the charge transfer in the image sensing device. The control circuit may include, for example, Figure 1 , 3 , circuits such as the pulse irradiation unit 110, the sensor unit 130, and the ToF timing generator 150 shown in 4A and 4B.

[0193] In some embodiments, the image sensor device can be used for differential ToF measurement. Figures 28 to 31 As described above, the control circuit in the image sensor device is configured to implement the method in differential ToF mode.

[0194] Fig.33 33 is a flow chart outlining a method 3300 for differential ToF mode operation. The method begins at 3310, where a pixel unit is exposed to light during an exposure time window. At 3320, during a first period, collected photocharge is transferred from a first pair of photodiodes to a first storage diode disposed between the first pair of photodiodes, and at 3330, the collected photocharge is transferred from a second pair of photodiodes to a second storage diode disposed between the second pair of photodiodes.

[0195] During the second period, at 3340, the collected photocharges are transferred from the third pair of photodiodes to a third storage diode disposed between the third pair of photodiodes, and at 3350, the collected photocharges are transferred from the fourth pair of photodiodes to a fourth storage diode disposed between the fourth pair of photodiodes. At 3360, a first sum of photocharges from the second storage diode and the fourth storage diode is formed. At 3370, a second sum of photocharges from the first storage diode and the third storage diode is formed. At 3380, the first sum is transmitted to a first input terminal of a differential amplifier, and the second sum is transmitted to a second input terminal of the differential amplifier. At 3390, the differential amplifier determines a differential signal based on the first sum and the second sum. The differential signal can be used to determine ToF.

[0196] It should be understood that Fig.33 The specific steps shown provide a specific method for performing differential ToF mode operation according to an embodiment of the present invention. According to alternative embodiments, other step sequences may also be performed. For example, alternative embodiments of the present invention may perform the steps listed above in a different order. In addition, Fig.33 The various steps shown may include multiple sub-steps that are performed in various orders suitable for the various steps. In addition, additional steps may be added or deleted depending on the specific application. Those of ordinary skill in the art will recognize many variations, modifications and substitutions.

[0197] In some embodiments, a method for operating a pixel unit in a differential ToF mode may include exposing the pixel unit to light during an exposure time window, as described above, the pixel unit including four photodiodes and four storage diodes. The method includes transferring the collected photocharge from a first pair of photodiodes to a first storage diode disposed between the first pair of photodiodes during a first period, and transferring the collected photocharge from a second pair of photodiodes to a second storage diode disposed between the second pair of photodiodes. The method includes transferring the collected photocharge from a third pair of photodiodes to a third storage diode disposed between the third pair of photodiodes during a second period, and transferring the collected photocharge from a fourth pair of photodiodes to a fourth storage diode disposed between the fourth pair of photodiodes. The method also includes generating a differential signal by providing the sum of the photocharges from the first storage diode and the second storage diode to a first input terminal of a differential amplifier, and providing the sum of the photocharges from the third storage diode and the fourth storage diode to a second input terminal of the differential amplifier.

[0198] In some embodiments, the method may further include transferring photocharge from the first storage diode and the second storage diode to the first floating diffusion region, and transferring photocharge from the first storage diode and the second storage diode to the second floating diffusion region. The method may further include transferring photocharge from the first floating diffusion region to the first sample-and-hold capacitor, transferring photocharge from the second floating diffusion region to the second sample-and-hold capacitor, and transmitting signals from the first and second sample-and-hold capacitors to a differential amplifier. In some embodiments of the method, the first pair of photodiodes and the second pair of photodiodes have no common photodiodes, and the third pair of photodiodes and the fourth pair of photodiodes have no common photodiodes. Example Combination of Methods for Differential ToF Mode Figures 28 to 31 Described.

[0199] In some embodiments, a method for operating a pixel cell in a binning mode may include transferring photocharge collected in a first pair of adjacent photodiodes to a first storage diode, and transferring photocharge collected in a second pair of adjacent photodiodes to a second storage diode. The method may also include sensing the photocharge in the first storage diode and the second storage diode to provide two sensing signals for binning. Example combinations of methods for binning mode Fig.34 Give a description.

[0200] In some embodiments, a method for operating a pixel unit in full resolution mode may include transferring photocharge collected in each photodiode to an adjacent storage diode, and sensing the photocharge in each storage diode to provide sensing signals for four sub-pixels. An example of a method for full resolution mode may include transferring photocharge collected in each photodiode to an adjacent storage diode, and sensing the photocharge in each storage diode to provide sensing signals for four sub-pixels. Fig.35 Give a description.

[0201] like Fig.28 As shown, the pixel unit 2800 includes four photodiodes PPD1, PPD2, PPD3 and PPD4. The pixel unit 2800 also includes four storage diodes, which are arranged under the four storage diode charge control gates. It can be seen that each storage diode is arranged between the first adjacent photodiode and the second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and / or the second adjacent photodiode. In addition, each photodiode is arranged between the first adjacent storage diode and the second adjacent storage diode, and each photodiode is configured to transfer the photocharges to one or both of the first adjacent storage diode and / or the second adjacent storage diode under the control of the transfer gate. The charge transfer between the photodiode and the storage diode is performed under the control of the transfer gates TX1_1, TX1_2, TX1_3 and TX1_4, as shown in combination with the above. Fig.28 described.

[0202] exist Figures 28 to 29B In an embodiment of the invention, four photodiodes are arranged in a 2×2 array, wherein each storage diode is disposed between two adjacent photodiodes. There is a transfer gate between each pair of adjacent photodiodes and storage diodes. Fig.28 Pixel unit 2800 in the embodiment of the present invention further comprises four floating diffusion regions. Each floating diffusion region is arranged adjacent to a corresponding storage diode. A floating diffusion region FD1 is shown. There is a transfer gate between each pair of adjacent storage diodes and floating diffusion regions. There is also a charge control gate TX3 above each storage diode, and the charge control gate TX3 is configured to maintain a constant voltage.

[0203] In some embodiments of the present invention, Figures 28 to 29B The pixel unit shown may be used for ToF distance measurement in an image sensor device.The image sensing device may include a plurality of pixel units arranged in a matrix in a pixel array; and a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Figure 2A and 2B An example of a plurality of pixel units arranged in a matrix in a pixel array is shown. The control circuit may include, for example, Figure 1 , Figure 3 as well as Figure 4A and 4B Circuits such as the pulse irradiation unit 110, the sensor unit 130 and the ToF timing generator 150 are shown.

[0204] Fig.34 is a simplified top view schematic diagram illustrating a pixel cell for binning mode operation according to some embodiments of the present invention. Fig.34 3400 including four photodiodes and four storage diodes, similar to Figures 28 to 29B Each storage diode is disposed between two adjacent photodiodes, and each storage diode is configured to receive photocharges from one or both of the two adjacent photodiodes. Each photodiode is disposed between two adjacent storage diodes, and each photodiode is configured to transfer photocharges to one or both of the two adjacent storage diodes. For the binning operation mode, photocharges collected in a first pair of adjacent photodiodes are transferred to a first storage diode, and photocharges collected in a second pair of adjacent photodiodes are transferred to a second storage diode. For example, in Fig.34 In the example, the photocharges collected in the photodiodes PPD2 and PPD3 are transferred to the storage diode SD3, and the photocharges collected in the photodiodes PPD1 and PPD4 are transferred to the storage diode SD1. Then, the photocharges in the first storage diode (SD3) and the second storage diode (SD1) are sensed to provide two sensing signals for binning. In this example, the transfer signals TX1_1 and TX1_2 are switched together to control the charge transfer. The above operation is similar to that in Fig.28 Alternatively, different photodiodes may be paired for binning. For example, similar to Fig.28 The operation that occurs during timing pulse #2 in the first embodiment transfers the photocharge collected in photodiodes PPD1 and PPD2 to storage diode SD2, and transfers the photocharge collected in photodiodes PPD3 and PPD4 to storage diode SD4. The photocharge in the first storage diode (SD2) and the second storage diode (SD4) is then sensed to provide two sense signals for binning. In this example, transfer signals TX1_3 and TX1_4 are switched together to control the charge transfer. As shown below with reference to Fig.37C As described, the binning mode may be used in binned differential mode ToF determination.

[0205] Fig.35 is a simplified top view schematic diagram illustrating a pixel cell for full resolution mode operation according to some embodiments of the present invention. Fig.35 3500 including four photodiodes and four storage diodes, similar to Figures 28 to 29B Each storage diode is disposed between two adjacent photodiodes, and each storage diode is configured to receive photocharges from one or both of the two adjacent photodiodes. Each photodiode is disposed between two adjacent storage diodes, and each photodiode is configured to transfer photocharges to one or both of the two adjacent storage diodes. For full-resolution mode operation, the photocharges collected in each photodiode are transferred to the corresponding adjacent storage diodes. The photocharges in each storage diode are sensed to provide sensing signals for four sub-pixels. For example, in Fig.35 In the example, the photocharge collected in the photodiode PPD1 is transferred to the storage diode SD2, the photocharge collected in the photodiode PPD2 is transferred to the storage diode SD3, the photocharge collected in the photodiode PPD3 is transferred to the storage diode SD4, and the photocharge collected in the photodiode PPD4 is transferred to the storage diode SD1. Then, the photocharge in each of the four storage diodes SD1, SD2, SD3 and SD4 is sensed to provide four sensing signals for each of the four sub-pixels. In this example, the transfer signals TX1_2 and TX1_4 can be switched together to control the charge transfer. Fig.35 Also shown are four readout circuits RO_1, RO_2, RO_3 and RO_4, each having a floating diffusion region for reading out an optical signal.

[0206] Alternatively, a different association of the photodiode and the storage diode can also be used for full resolution mode operation. Fig.35 In the embodiment, the photocharges collected in the photodiode PPD1 may be transferred to the storage diode SD1, the photocharges collected in the photodiode PPD2 may be transferred to the storage diode SD2, the photocharges collected in the photodiode PPD3 may be transferred to the storage diode SD3, and the photocharges collected in the photodiode PPD4 may be transferred to the storage diode SD4. Then, the photocharges in each of the four storage diodes SD1, SD2, SD3, and SD4 are sensed to provide four sensing signals for each of the four sub-pixels.

[0207] Fig.36 is a diagram showing some embodiments according to the present invention Fig.28 A line drawing of the layout of a portion of a pixel cell. Fig. 30A In the timing diagram, in differential ToF mode, the two swing phases (such as Fig.31 A) are interleaved between two rows of pixels to reduce edge capacitance. The two metal lines 3610 and 3612 associated with the label "Storage 1" (e.g., with Fig.28 3 and 4) are switched together. Therefore, the capacitance between the two lines is low. Similarly, the two metal lines 3620 and 3622 associated with the tag "Storage 2" (e.g., Fig.28 The metal lines associated with the transfer signals TX1_1 and TX1_2 in the memory are switched together. Therefore, the capacitance between the two lines is low. On the other hand, the two metal lines associated with the tag "Storage 1" are set away from the two metal lines associated with the tag "Storage 2". This arrangement can reduce the capacitance even when thick metal lines are used to reduce resistance. This arrangement is ideal for ToF operation when a fast response is required, such as in the sub-nanosecond range.

[0208] Fig.37A , 37B and 37C are diagrams showing how some embodiments of the present invention may be used. Figures 28 to 29B Simplified timing diagram of the different operating modes implemented by the pixel unit. Fig.37A A timing diagram for a standard image sensor differential mode (where two full readouts are required) is shown. In this mode, the first pulse operates with active illumination and the second pulse operates when the active illumination is in the "off" state. In this differential mode, the sensor operates in binning mode and a full readout is performed between two captures. The differential signal can be obtained by digitally subtracting between the captured frames.

[0209] Fig.37B The timing diagram of the intra-pixel differential ToF mode is shown, as shown above in conjunction with Figures 28 to 36 , where only one differential readout is performed. In this mode, there is no readout sequence between the actively illuminated subframe and the off-state subframe. In this intra-pixel differential mode, the sensor operates in differential mode and the differential signal is generated by subtraction performed using a differential amplifier.

[0210] Fig.37C A timing diagram of a high-level "dynamic mode" operation according to some embodiments of the present invention is shown. In dynamic mode, an array of pixel cells is scanned with a differential comparator, and the addresses of pixels with sense signals above a preset threshold are sent over a communication link such as an SPI (serial peripheral interface). In this mode, selective, event-based outputs can be provided at low data bandwidth. In some embodiments, active illumination is not required, and signal changes over time are tracked by alternately integrating into two stores. In some embodiments, each pixel has two stores, and an image can be compared to a previous image. The difference can be tested with a rolling shutter and a simplified ADC (analog-to-digital converter) containing only a single bit. This can be implemented using a comparator.

[0211] Although the preferred embodiments of the present invention have been shown and described, it will be clear that the present invention is not limited to only these embodiments. Many modifications, changes, variations, substitutions and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the present invention described in the claims.< / j> < / j> < / j> < / j> < / j> < / j> < / j> < / j> < / j> < / j>

Claims

1. A method of operating a pixel circuit for time-of-flight (ToF) distance measurement, wherein the pixel circuit comprises: Photodiode; a drain region adjacent to the photodiode and coupled to a bias voltage; a shutter gate disposed between the photodiode and the drain region, the shutter gate being controlled by a global shutter signal to apply the bias voltage to bias the photodiode for light sensing; a storage diode coupled to the photodiode through a first transfer gate controlled by a first transfer signal; as well as a floating diffusion region coupled to the storage diode through a second transfer gate controlled by a second transfer signal; The method comprises: During the exposure period, activating the first transfer gate using the first transfer signal to couple the photodiode and the storage diode; as well as activating the photodiode in a first plurality of time windows to sense light reflected from the target due to a corresponding plurality of emission light pulses, wherein a delay time between each time window and a corresponding emission light pulse is designated as D1; as well as During the sampling period: The second transfer gate is activated using the second transfer signal to transfer charge from the storage diode to the floating diffusion region, wherein the charge in the floating diffusion region is sampled to determine a first sampling signal S1 representing the charge collected during the exposure period.

2. The method according to claim 1, wherein: Exposing the photodiode in a first plurality of time windows includes providing a first global shutter signal to the shutter gate, the first global shutter signal having a first plurality of global shutter pulses to expose the photodiode.

3. The method according to claim 1, wherein: Exposing the photodiode in a first plurality of time windows includes coupling the drain region adjacent to the photodiode to a plurality of bias voltage pulses (VAAGS).

4. The method according to claim 1, wherein: The bias voltage is higher than the first transfer signal and the second transfer signal.

5. The method according to claim 1, wherein: The charge in the floating diffusion region is sampled using correlated double sampling (CDS).

6. A method of operating a pixel circuit for time-of-flight (ToF) distance measurement, wherein the pixel circuit comprises: Photodiode; a drain region adjacent to the photodiode and coupled to a bias voltage; a shutter gate disposed between the photodiode and the drain region, the shutter gate being controlled by a global shutter signal to apply the bias voltage to bias the photodiode for light sensing; a storage diode coupled to the photodiode through a first transfer gate controlled by a first transfer signal; as well as a floating diffusion region coupled to the storage diode through a second transfer gate controlled by a second transfer signal; The method comprises: In the first exposure period, activating the first transfer gate using the first transfer signal to couple the photodiode and the storage diode; activating the photodiode in a first plurality of time windows to sense light reflected from the target due to a corresponding first plurality of transmitted light pulses, wherein a delay time between each time window and a corresponding transmitted light pulse is designated as D1; In the first sampling period: activating the second transfer gate using the second transfer signal to transfer charge from the storage diode to the floating diffusion region, wherein the charge in the floating diffusion region is sampled to determine a first sampling signal S1 representing the charge collected during the first exposure period, In the second exposure period, activating the first transfer gate using the first transfer signal to couple the photodiode and the storage diode; and activating the photodiode in a second plurality of time windows to sense light reflected from the target due to a corresponding second plurality of transmitted light pulses, wherein a delay time between each time window and a corresponding transmitted light pulse is designated as D2; In the second sampling period: activating the second transfer gate using the second transfer signal to transfer charge from the storage diode to the floating diffusion region, wherein the charge in the floating diffusion region is sampled to determine a second sampling signal S2 representing the charge collected during the second exposure period; and The distance to the target is determined based on the first sampling signal S1 and the second sampling signal S2.

7. The method according to claim 6, wherein: Determining the distance to the target includes using a lookup table based on the first sampled signal S1 and the second sampled signal S2.

8. The method according to claim 6, wherein: Determining the distance to the target includes using a convolution method.

9. The method according to claim 6, wherein: Exposing the photodiode in a first plurality of time windows includes providing a first global shutter signal to the shutter gate, the first global shutter signal having a first plurality of global shutter pulses to expose the photodiode.

10. The method according to claim 6, wherein: Exposing the photodiode in a first plurality of time windows includes coupling the drain region of the photodiode to a plurality of bias voltage pulses (VAAGS).

11. The method according to claim 6, wherein: The bias voltage is higher than the first transfer signal and the second transfer signal.

12. The method according to claim 6, wherein: The charge in the floating diffusion region is sampled using correlated double sampling (CDS).

13. An image sensor device comprising: a plurality of pixel units arranged in a pixel array; a control circuit for controlling an exposure phase and a sampling phase of the image sensor device; as well as a switch circuit for coupling a pixel power supply line to a first voltage in an exposure phase and coupling the pixel power supply line to a second voltage in a sampling phase, the first voltage being higher than the second voltage; Each of the plurality of pixel units comprises: a photodiode in the semiconductor substrate, a first end of the photodiode coupled to a bias voltage through a shutter gate controlled by a global shutter signal; a ground contact for coupling the second end of the photodiode to electrical ground via an electrical ground conductor; a storage diode located in the semiconductor substrate and coupled to the second end of the photodiode through a first transfer gate controlled by a first transfer signal; and A floating diffusion region is located in the semiconductor substrate and is coupled to the storage diode through a second transfer gate controlled by a second transfer signal.

14. The image sensor device according to claim 13, wherein: The control circuit is configured to activate the photodiode in a plurality of time windows to sense light reflected from the target due to the corresponding plurality of emission light pulses by a predetermined delay time between each time window and the corresponding emission light pulse.

15. The image sensor device of claim 14, further comprising a drain region, wherein: The first end of the photodiode is coupled to a bias voltage through the drain region; as well as The control circuit is configured to provide a plurality of bias voltage pulses to activate the photodiode in the plurality of time windows to sense reflected light.

16. The image sensor device according to claim 14, wherein: The control circuit is configured to provide a plurality of global shutter signal pulses to activate the photodiode in the plurality of time windows to sense reflected light.

17. A pixel unit for time-of-flight (ToF) distance measurement, comprising: substrate; Photodiode; a ground contact for coupling the second end of the photodiode to electrical ground via an electrical ground conductor; a drain region adjacent to the photodiode and coupled to a bias voltage; a shutter gate disposed between the photodiode and the drain region, the shutter gate being controlled by a global shutter signal to apply the bias voltage to bias the photodiode for light sensing; a storage diode coupled to the photodiode through a first transfer gate controlled by a first transfer signal; as well as A floating diffusion region is coupled to the storage diode through a second transfer gate controlled by a second transfer signal.

18. The pixel unit according to claim 17, wherein: The photodiode is formed in a rectangular diffusion region having four side lines; The shutter gate is disposed along a first side line between the photodiode and the drain region, and the shutter gate has the same length as the first side line; and The first transfer gate is disposed along a second side line adjacent to the first side line, a length of the first transfer gate is half a length of the second side line, and the first transfer gate is disposed above a corner of the photodiode away from the first side line.

19. The pixel unit according to claim 17, further comprising a first metal interconnection layer, a second metal interconnection layer and a third metal interconnection layer, wherein: A first transmission line, a second transmission line, a select line, and a reset line are formed in the first metal interconnect layer along a first direction and span a total width W; VDDA lines, VSSA lines and pixel output lines are formed in the second metal interconnect layer along a second direction perpendicular to the first direction; as well as A global shutter line is formed in the third metal interconnection layer along the first direction, the global shutter line having a width substantially equal to W.

20. The pixel unit according to claim 17, wherein: The drain region includes a low capacitance drain diffusion region to facilitate modulation of the bias voltage.

21. A pixel unit, comprising: Four photodiodes; as well as four storage diodes; wherein each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and the second adjacent photodiode; and Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharges to one or both of the first adjacent storage diode and the second adjacent storage diode.

22. The pixel unit according to claim 21, wherein: The four photodiodes are arranged in a 2×2 array, where each storage diode is disposed between two adjacent photodiodes.

23. The pixel cell of claim 21, further comprising a transfer gate between each adjacent pair of the photodiode and the storage diode.

24. The pixel cell of claim 21, further comprising a charge control gate located above each storage diode.

25. The pixel cell of claim 21, further comprising four floating diffusion regions, each floating diffusion region being disposed adjacent to a corresponding storage diode.

26. The pixel cell of claim 25, further comprising a transfer gate located between each pair of adjacent storage diodes and floating diffusion regions.

27. The pixel cell of claim 21, further comprising a global shutter control gate associated with each photodiode for draining charge in the photodiode.

28. An image sensor device comprising a plurality of pixel units according to claim 21 arranged in a pixel array.

29. A method for operating a pixel unit, the method comprising: exposing a pixel unit to light during an exposure time window, the pixel unit comprising four photodiodes and four storage diodes; wherein each storage diode is disposed between two adjacent photodiodes, and each storage diode is configured to receive photocharges from one or both of the two adjacent photodiodes; and Each photodiode is disposed between two adjacent storage diodes, and each photodiode is configured to transfer photocharges to one or both of the two adjacent storage diodes.

30. The method of claim 29, further comprising: During the first period, transferring the collected photocharges from the first pair of photodiodes to a first storage diode disposed between the first pair of photodiodes; as well as transferring the collected photocharges from the second pair of photodiodes to a second storage diode disposed between the second pair of photodiodes; During the second period, transferring the collected photocharges from the third pair of photodiodes to a third storage diode disposed between the third pair of photodiodes; as well as transferring the collected photocharges from the fourth pair of photodiodes to a fourth storage diode disposed between the fourth pair of photodiodes; as well as The differential signal is generated by: providing a sum of the photocharges from the first storage diode and the second storage diode to a first input terminal of a differential amplifier; and A sum of the photocharges from the third storage diode and the fourth storage diode is provided to a second input terminal of the differential amplifier.

31. The method of claim 30, further comprising: transferring photocharges from the first storage diode and the second storage diode to a first floating diffusion region; transferring photocharges from the first storage diode and the second storage diode to a second floating diffusion region; transferring photocharge from the first floating diffusion region to a first sample-and-hold capacitor; transferring the photocharge from the second floating diffusion region to a second sample-and-hold capacitor; as well as A signal is transmitted from the first sample-and-hold capacitor and the second sample-and-hold capacitor to the differential amplifier.

32. The method of claim 30, wherein: The first pair of photodiodes and the second pair of photodiodes have no photodiodes in common; and The third pair of photodiodes and the fourth pair of photodiodes have no common photodiode.

33. The method of claim 29, further comprising: transferring the photocharges collected in the first pair of adjacent photodiodes to the first storage diode; transferring the photocharges collected in the second pair of adjacent photodiodes to a second storage diode; as well as Photocharges in the first storage diode and the second storage diode are sensed to provide two sense signals for binning.

34. The method of claim 29, further comprising: Transferring the photocharge collected in each photodiode to the corresponding adjacent storage diode; as well as The photocharge in each storage diode is sensed to provide sense signals for four sub-pixels.

35. A pixel unit for differential light sensing, comprising: Multiple photodiodes; as well as a corresponding plurality of storage diodes; wherein each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and the second adjacent photodiode; and Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharges to one or both of the first adjacent storage diode and the second adjacent storage diode.

36. The pixel cell of claim 35, further comprising a corresponding plurality of floating diffusion regions, each floating diffusion region being disposed adjacent to a corresponding storage diode.

37. An image sensing device comprising: A plurality of pixel units arranged in a pixel array, wherein each pixel unit comprises four photodiodes, four storage diodes and four floating diffusion regions, wherein: Each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photocharges from one or both of the first adjacent photodiode and the second adjacent photodiode; Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photocharge to one or both of the first adjacent storage diode and the second adjacent storage diode; and Each floating diffusion region is disposed adjacent to a corresponding storage diode; a first summing device for receiving photocharges from the second floating diffusion region and the fourth floating diffusion region; a second summing device for receiving photocharges from the first floating diffusion region and the third floating diffusion region; a differential amplifier coupled to the first summing device and the second summing device; and A control circuit is used to control the charge transfer in the image sensing device.

38. The image sensing device according to claim 37, wherein: The control circuit is configured to: exposing the pixel unit to light during an exposure time window; During the first period, transferring the collected photocharges from the first pair of photodiodes to a first storage diode disposed between the first pair of photodiodes; as well as transferring the collected photocharges from the second pair of photodiodes to a second storage diode disposed between the second pair of photodiodes; During the second period, transferring the collected photocharges from the third pair of photodiodes to a third storage diode disposed between the third pair of photodiodes; as well as transferring the collected photocharges from the fourth pair of photodiodes to a fourth storage diode disposed between the fourth pair of photodiodes; as well as The differential signal is generated by: providing a sum of the photocharges from the second storage diode and the fourth storage diode to a first input terminal of the differential amplifier; and A sum of the photocharges from the first storage diode and the third storage diode is provided to a second input terminal of the differential amplifier.

39. The image sensing device according to claim 37, wherein: The control circuit is configured to: exposing the pixel unit to light during an exposure time window; transferring the photocharges collected in the first pair of adjacent photodiodes to the first storage diode; transferring the photocharges collected in the second pair of adjacent photodiodes to a second storage diode; as well as Photocharges in the first storage diode and the second storage diode are sensed to provide two sense signals for binning.

40. The image sensing device according to claim 37, wherein: The control circuit is configured to: exposing the pixel unit to light during an exposure time window; Transferring the photocharge collected in each photodiode to the corresponding adjacent storage diode; as well as The photocharge in each storage diode is sensed to provide sense signals for four sub-pixels.

41. An image sensing device, comprising: Photodiode array; a first storage diode disposed between a first pair of photodiodes in the array, wherein the first storage diode is configured to receive photocharges from each photodiode in the first pair of photodiodes; a first floating diffusion region disposed adjacent to the first storage diode; a second storage diode disposed between a second pair of photodiodes in the array, wherein the second storage diode is configured to receive photocharges from each photodiode in the second pair of photodiodes; a second floating diffusion region disposed adjacent to the second storage diode; as well as A circuit is configured to receive photo charges from the first floating diffusion area and the second floating diffusion area.

42. The image sensing device according to claim 41, wherein: The circuit is configured to sense photocharges in the first storage diode and the second storage diode to provide two sense signals for binning.

43. The image sensing device according to claim 41, wherein: The circuit is configured to sum the photocharges in the first storage diode and the second storage diode.

44. The image sensing device according to claim 41, further comprising: a third storage diode disposed between a third pair of photodiodes in the array, wherein the third storage diode is configured to receive photocharges from each photodiode in the third pair of photodiodes; a third floating diffusion region, which is disposed adjacent to the first storage diode; a fourth storage diode disposed between a fourth pair of photodiodes in the array, wherein the fourth storage diode is configured to receive photocharges from each photodiode in the fourth pair of photodiodes; a fourth floating diffusion region, which is disposed adjacent to the fourth storage diode; as well as The circuit is further configured to receive photo charges from the third floating diffusion area and the fourth floating diffusion area.

45. The image sensing device according to claim 44, wherein: The circuit is further configured to: summing the photocharges in the first storage diode and the second storage diode; summing the photocharges in the third storage diode and the fourth storage diode; as well as A sum of the photocharges in the first storage diode and the second storage diode is compared with a sum of the photocharges in the third storage diode and the fourth storage diode.

46. ​​The image sensing device of claim 44, wherein: The first pair of photodiodes and the second pair of photodiodes have no photodiodes in common; and The third pair of photodiodes and the fourth pair of photodiodes have no common photodiode.

Images