Device for capturing 2D images and depth images of a scene

Abstract

The invention relates to a device for acquiring a 2D image and a depth image of a scene, comprising depth photosensitive cells (Z) and capacitors (Cd), each depth photosensitive cell comprising a photodiode (PH) able to detect a reflected light signal, and at least one sensing node (SN, SN') coupled to the photodiode through a single transistor (44, 64). Each capacitor is connected between two sensing nodes of two photosensitive cells or between two sensing nodes of the same photosensitive cell. The depth photosensitive cell provides at least one first sample of charge generated by light during a first time period to a first plate of each capacitor and a second sample of charge generated by light during a second time period to a second plate of each capacitor. The depth photosensitive cell provides at least one third sample of charge generated by light during a third time period to the first plate of each capacitor.

Description

Technical Field

[0001] This application relates to image acquisition devices, and more specifically, to image acquisition devices capable of acquiring 2D images and depth images of a scene. Background Technology

[0002] Image acquisition devices capable of acquiring depth information have already been provided. For example, an indirect time-of-flight (iToF) detector works by emitting a light signal into a scene and then detecting the reflected light signal from objects in the scene. By estimating the phase shift between the emitted and reflected light signals, the distance to the image acquisition device in the scene can be estimated.

[0003] In some applications, it is desirable to be able to capture a 2D image of a scene and a corresponding depth image of the scene.

[0004] While a solution to achieve this goal is to use separate image sensors to capture 2D and depth images, this is not optimal because the sensors will have different viewpoints on the scene, leading to misalignment between pixels in the corresponding images. Furthermore, using two sensors increases the size and cost of the device.

[0005] Another solution is to integrate the pixels of the 2D image and the depth pixels in the same array of the detector. However, it is then desirable to reduce the size of the depth pixels to a value that minimizes the loss of information about the 2D image.

[0006] A device is desired that acquires 2D and depth images of a scene, overcoming at least partially one or more drawbacks of known devices. Summary of the Invention

[0007] An embodiment provides an apparatus for acquiring depth images and 2D images of a scene, including 2D image pixels, depth photosensitive units, and capacitors. Each capacitor includes a first plate and a second plate. Each depth photosensitive unit includes a photodiode capable of detecting a reflected light signal corresponding to the reflection of a near-infrared incident amplitude-modulated light signal on the scene, and at least one sensing node coupled to the photodiode via a single transistor. Each capacitor is connected between sensing nodes of two photosensitive units or between two sensing nodes of the same photosensitive unit. At least a portion of the depth photosensitive unit is configured to: provide at least one first sample of charge (generated by detecting the reflected light signal during a first time period) to the first plate of each capacitor of at least a portion of the capacitor, and provide a second sample of charge (generated by detecting the reflected light signal during a second time period) to the second plate of each capacitor of said portion of the capacitor, the second time period being offset by a first constant phase shift relative to the first time period. At least a portion of the depth photosensitive unit is configured to provide at least a third sample of charge (generated by detecting the reflected light signal during a third time period) to the first plate of each capacitor of at least a portion of the capacitor, the third time period being offset by a second constant phase shift different from the first phase shift relative to the first time period.

[0008] According to an embodiment, the first sample and the second sample are provided by the same depth photosensitive unit.

[0009] According to an embodiment, the first sample and the second sample are provided by photosensitive units at different depths.

[0010] According to an embodiment, the depth sensing unit of the portion of the depth sensing unit is configured to provide a fourth sample of charge (which is photogenerated by detecting reflected light signals during a fourth time period) to the second plate of each capacitor of the portion of the capacitor, the fourth time period being offset relative to the first time period by a third constant phase shift that is different from the first phase shift and the second phase shift.

[0011] According to an embodiment, the third and fourth samples are provided by the same depth photosensitive unit.

[0012] According to an embodiment, the first sample and the second sample are provided by the same depth photosensitive unit, and the third sample and the fourth sample are provided by a different depth photosensitive unit than the depth photosensitive unit that provided the third sample and the fourth sample.

[0013] According to an embodiment, the first sample and the second sample are provided by the same photosensitive unit during the acquisition of the first depth image, and the third sample and the fourth sample are provided by the same depth photosensitive unit that provided the first sample and the second sample during the acquisition of the second depth image.

[0014] According to an embodiment, the third and fourth samples are provided by photosensitive units at different depths.

[0015] According to an embodiment, the first sample and the second sample are provided by different depth photosensitive units, and the third sample and the fourth sample are provided by different depth photosensitive units than the depth photosensitive units that provided the third sample and the fourth sample.

[0016] According to an embodiment, the first sample and the second sample are provided by different depth photosensitive units during the acquisition of the first depth image, and the third sample and the fourth sample are provided by the depth photosensitive unit that provided the first sample and the second sample during the acquisition of the second depth image.

[0017] The embodiments also provide a system for acquiring depth images, including: an acquisition device such as those previously defined; a light source configured to emit near-infrared periodically amplitude-modulated incident light signals; and a processor configured to determine the phase shift between the incident light signals and reflected light signals from a first sample, a second sample, and a third sample.

[0018] According to an embodiment, the processor is configured to determine the phase shift between the incident light signal and the reflected light signal from the first sample, the second sample, the third sample, and the fourth sample. Attached Figure Description

[0019] The above-described features and advantages, as well as other features and advantages, will be described in detail below with reference to the accompanying drawings, which are given by way of illustration rather than limitation, in which:

[0020] Figure 1 An embodiment of a 2D image and depth image forming system is illustrated schematically;

[0021] Figure 2 This is a graph showing an example of the light intensity of the emitted and reflected light signals according to an embodiment.

[0022] Figure 3 An example of the circuitry for a photosensitive unit capable of detecting depth information is shown schematically;

[0023] Figure 4 Another example of the circuitry for a photosensitive unit capable of detecting depth information is illustrated schematically;

[0024] Figure 5 It is a timing diagram of the voltage at the sensing node of the photosensitive unit during the readout operation;

[0025] Figure 6 The principle of a known depth pixel readout method is illustrated;

[0026] Figure 7 The principle of an embodiment of the differential depth pixel readout method is shown;

[0027] Figure 8An embodiment of the arrangement of photosensitive units in a depth image acquisition device is schematically illustrated;

[0028] Figure 9 Another embodiment of the arrangement of photosensitive units in a depth image acquisition device is schematically shown;

[0029] Figure 10 Another embodiment of the arrangement of photosensitive units in a depth image acquisition device is schematically shown;

[0030] Figure 11 Another embodiment of the arrangement of photosensitive units in a depth image acquisition device is schematically shown;

[0031] Figure 12A The steps of an embodiment of the differential readout method are shown;

[0032] Figure 12B Another step of the method is shown;

[0033] Figure 12C Another step of the method is shown;

[0034] Figure 12D Another step of the method is shown;

[0035] Figure 13 The steps of another embodiment of the differential readout method are shown;

[0036] Figure 14 An embodiment of the circuitry for a photosensitive unit capable of detecting depth information is schematically illustrated; and

[0037] Figure 15 Another embodiment of the circuitry for a photosensitive unit capable of detecting depth information is illustrated schematically. Detailed Implementation

[0038] In the various figures, similar features are designated by similar reference numerals. In particular, common structural and / or functional features in various embodiments may have the same reference numerals and may be arranged with the same structure, dimensions, and material properties.

[0039] For clarity, only steps and elements useful for understanding the embodiments described herein are shown and described in detail. In particular, the formation of photodiodes for 2D image pixels and depth pixels is not described in detail; such pixel formation is within the capabilities of those skilled in the art based on the indications in this specification. Unless otherwise stated, when referring to two elements connected together, this means there is no direct connection of any intermediate element other than a conductor, and when referring to two elements coupled together, this means that the two elements can be connected or they can be coupled via one or more other elements.

[0040] In the following description, when referring to terms defining absolute position, such as "front," "back," "top," "bottom," "left," "right," etc., or relative position, such as "above," "below," "upper," "lower," etc., or when referring to terms defining direction, such as "horizontal," "vertical," etc., it refers to the orientation of the accompanying drawings or the depth image acquisition device in its normal use position. Unless otherwise stated, the expressions "about," "approximately," "substantially," and "about" indicate within 10%, and preferably within 5%.

[0041] Figure 1 An embodiment of a depth image forming system 10 is schematically illustrated, comprising a device 12 for acquiring a combined 2D image and a depth image. System 10 includes, for example, a light signal emitting circuit 14 that drives a light source 16, such as a light-emitting diode (LED). The LED 16 emits a light signal, for example, at wavelengths in the near-infrared spectrum, e.g., in the range from 700 nm to 1,100 nm. The light signal generated by the LED 16 is transmitted, for example, via one or more lenses (…). Figure 1 The light (not shown) is emitted toward the image scene Sc. The light signal reflected from the image scene Sc is captured by the image acquisition device 12, for example, via the imaging lens 17 and the microlens array 18, which focuses the light onto the individual pixels of the image acquisition device 12.

[0042] For example, the image acquisition device 12 includes a plurality of pixels capable of receiving light signals reflected from an image scene and detecting the phase of the received signals to form a depth image. These pixels are hereinafter referred to as depth pixels. The image acquisition device 12 also includes a plurality of pixels capable of capturing visible light from the image scene to form a 2D image, these pixels are hereinafter referred to as 2D image pixels.

[0043] The processor 20 of the imaging system 10 is coupled to, for example, the image acquisition device 12 and the light signal emitting circuit 14, and determines the corresponding distance of objects in the image scene Sc based on the signals captured by the depth pixels of the image acquisition device 12. The 2D image captured by the image acquisition device 12 and the depth image generated by the processor 20 are stored, for example, in the memory 22 of the imaging system 10.

[0044] Figure 2 This is a graph, with curve 30 showing an example of the time-varying intensity of the light signal emitted by the light-emitting diode 16 into the image scene Sc, and curve 32 showing an example of the time-varying intensity of the light signal received by one of the depth pixels of the image acquisition device 12. Although for the sake of simplified comparison, these signals... Figure 2The image shows signals with the same intensity, but in reality, the intensity of the light signal received by each depth pixel may be significantly lower than the intensity of the emitted signal. Figure 2 In the example, the optical signal has a sine wave shape. However, in alternative embodiments, it can have different periodic shapes, such as a sum of sine waves, a triangle, or a square.

[0045] The depth pixels in this specification are based on a photodetector used to detect the phase of the received optical signal. There is a phase shift between the emitted and received optical signals. This represents the time of flight (ToF) from the light signal originating from the LED 16 to the object in the image scene Sc via the reflected light signal, reaching the image acquisition device 12. Therefore, an estimate of the distance d to the object in the image scene Sc can be calculated using the following equation:

[0046] [Math_1]

[0047]

[0048] Where c specifies the speed of light, and f specifies the frequency of the light signal.

[0049] Phase shift For example, estimation is based on sampling of the signal captured by depth pixels during at least three different sampling windows (preferably four different sampling windows) in each period of the optical signal. The technique based on detecting four samples per period is described in further detail in R. Lange and P. Seitz’s publication, “Solid-state TOF range camera,” IEEJ. on Quantum Electronics, Vol. 37, No. 3, March 2001, which is incorporated herein by reference under legal authorization. As an example, in Figure 2 The image shows the collection of four samples per cycle.

[0050] The samples in each sampling window are integrated over a large number of periods, such as approximately 100,000 periods, or more generally between 10,000 and 10 million periods. For example, each sampling window has a duration ranging from a quarter of the optical signal period. These sampling windows... Figure 2 They are referred to as C0, C1, C2, and C3, and in Figure 2In the example, each sampling window has the same duration, and the total cycle time of the four sampling windows is equal to the period of the optical signal. More generally, there may or may not be a time interval separating the sampling window from the next sampling window, and in some cases, there may be overlap between sampling windows. With four samples captured per cycle for the depth pixel, each sampling window has a duration, for example, ranging from 15% to 35% of the period of the optical signal. With three samples captured per cycle for the depth pixel, each sampling window has a duration, for example, ranging from 25% to 40% of the period of the optical signal.

[0051] The timing of the sampling windows C0 to C3 is controlled to be synchronized with the timing of the transmitted optical signal. For example, the optical signal transmitting circuit 14 is based on the clock signal CLK ( Figure 1 The optical signal is generated, and the image acquisition device 12 receives the same clock signal CLK to control the end time of each sampling window by introducing an appropriate phase shift, for example, using a delay element.

[0052] Based on the integral samples of the optical signal, and for a pure sinusoidal light wave, the phase shift of the optical signal is determined by acquiring four samples C0, C1, C2, and C3 per cycle. This can be determined using the following equation, Math_2:

[0053] [Math_2]

[0054]

[0055] Phase shift of the optical signal when three samples C0, C1, and C2 are acquired in each cycle. This can be determined using the following equation, Math_3:

[0056] [Math_3]

[0057]

[0058] In some embodiments, the frequency f of the optical signal is 25 MHz, or more generally in the range of 20 MHz to 200 MHz.

[0059] In the following description, a "photosite" refers to a photodetector and all electronic components (desired to be a depth image) that absorb light signals reflected from the scene through the photodetector to acquire at least one sample of the generated charge; and a "pixel" refers to all electronic components necessary to determine the depth value. Specifically, a pixel may include multiple photosites.

[0060] To determine the phase shift between the emitted light signal and the light signal received by the depth pixel. The received optical signal is sampled by continuously transmitting the following at regular intervals: the photogenerated charge in the photosensitive element of the photosensitive unit during the first sampling window C0; the photogenerated charge in the photosensitive element of the same photosensitive unit or another photosensitive unit during the second sampling window C1; (in the case of determining the depth value from three samples) the photogenerated charge in the photosensitive element of the same photosensitive unit or another photosensitive unit during the third sampling window C2; and (in the case of determining the depth value from four samples) the photogenerated charge in the photosensitive element of the same photosensitive unit or another photosensitive unit during the fourth sampling window C3. These three or four transmissions are repeated a large number of times, for example, 100,000 times, before the obtained signal is read by the output circuit.

[0061] Figure 3 This is a circuit diagram illustrating an example of circuitry 40 of a depth-sensing unit. Circuitry 40 is capable of performing voltage storage. Circuitry 40 includes a photosensitive element PH, such as a fast photodiode, coupled between node 42 and a source (e.g., ground) with a low reference potential. Node 42 is coupled to a sensing node SN via a transfer gate 44, which is, for example, an n-channel MOS transistor. Transfer gate 44 is controlled by a signal Vsn applied to its control node. Figure 3 As shown, a capacitor C can be added to the sensing node SN to increase the storage capacitance and reduce the thermal noise present in the photosensitive unit. As a variation, the capacitance at the sensing node SN can also be formed solely by the interconnection of the photosensitive units.

[0062] Circuit 40 also includes an output circuit formed by a follower source transistor 46, a select transistor 48, and a reset transistor 50, such as an n-channel MOS transistor. A sensing node SN is coupled to a control node of transistor 46, for example, its drain is coupled to a source at a high reference potential Vdd, and its source is coupled to the output line 52 of circuit 40 via transistor 48, which is controlled by a signal Vsel applied to its gate. Sensing node SN' is also coupled to a source at the reference potential Vdd via transistor 50, or to a source at another reference potential, which is controlled by a signal Vres applied to its gate. In alternative embodiments, the output circuitry may be shared by multiple photosensitive units, wherein the sensing node SN is, for example, coupled to a sampling circuit of one or more adjacent photosensitive units.

[0063] Sample reading is performed by transferring the charge collected in photodiode PH to sensing node SN. No storage is performed in the intermediate memory interpolated between photodiode PH and transmission gate 44 and coupled to photodiode PH via an additional transmission gate.

[0064] Circuit 40 also includes transistor 54, such as an n-channel MOS transistor, which couples node 42 to a source at a reference potential Vdd and enables photodiode PH to be reset. PH Control. Therefore, it enables control of the exposure time of photodiode PH by ensuring that photodiode PD is cleared before the synchronous integration of all sensor photodiode PH begins, and it enables anti-blooming functionality to prevent photodiode PH from overflowing into sensing node SN during normal array readout.

[0065] Reading from depth pixels is performed row by row by reading the values ​​stored on the sensing nodes SN of the row pixels during sampling. For the entire read, these values ​​must remain stable on the sensing nodes SN; however, the pixels always receive parasitic light. This can cause the sensing nodes SN waiting to be read to saturate due to overflow of the photodiode PH.

[0066] Figure 4 This is a circuit diagram illustrating another example of a depth sensing unit circuit 60. Circuit 60 includes all the elements of circuit 40 and includes a second sensing node SN' coupled to node 42 via a transmission gate 64, which is, for example, an n-channel MOS transistor. Transmission gate 64 is controlled by a signal Vsn' applied to its control node. Circuit 60 also includes a second output circuit formed by a follower source transistor 66, a select transistor 68, and a reset transistor 70, which are, for example, n-channel MOS transistors. Sensing node SN' is coupled to the control node of transistor 66, for example, its drain is coupled to the source of a reference potential Vdd, and its source is coupled to the output line 72 of circuit 60 via transistor 68, which is controlled by a signal Vsel' applied to its gate. Sensing node SN' is also coupled to the source of the power supply voltage Vdd via transistor 70, which is controlled by a signal Vres' applied to its gate.

[0067] Figure 5 During the read operation Figure 3 The voltage V at the reference ground at the sensing node SN of the circuit 40 shown in the figure. SN The timing diagram is shown. Time intervals t1, t2, and t3 are consecutive. Before time t1, a reset step is performed by turning on reset transistor 50 to obtain the reference value V1 at node SN. Between times t1 and t2, transistor 44 turns on and off several times consecutively, causing the charge collected by photodiode PH at each sampling to be transferred to sensing node SN, thus adjusting its voltage V. SNThe voltage level changes from reference value V1 to level V2. Between times t2 and t3, the voltage level at sensing node SN is read by turning on transistor 48. At time t3, a reset step is performed by turning on reset transistor 50 to bring the voltage at node SN to the reference value V1' corresponding to the start of a new read cycle, where the reference value V1' may differ from the reference value V1 due to thermal noise. It is desirable to use the difference between voltage levels V1 and V2 to eliminate the reference value V1. In practice, using the difference V1'-V2, the read of level V1' is performed shortly after the read of level V2. During the reset step, a noise level is set such that the noise level at reference value V1' is different from the noise level at value V2. With a capacitance of approximately 5fF, the thermal noise is approximately 1mV.

[0068] In the case of 2D image pixels, this noise can be interfering. This is why it's not desirable to use methods such as combining... Figure 3 and Figure 4 The reason for the described pixel structure. The inventors have shown that for depth pixels that capture radiation reflected corresponding to near-infrared mid-amplitude modulation signals, this thermal noise will not be an interference, because infrared background light is present anyway, which adds noise to the measurement of the reflected light signal.

[0069] According to the embodiment, instead of reading the integral values ​​of samples C0, C1, C2, and C3, device 12 is configured to directly read the difference between two integral samples. In fact, as shown in equations Math_2 and Math_3, depth estimation uses only the difference between samples.

[0070] Figure 6 and Figure 7 The principles of conventional sample reading and differential reading are illustrated (very schematically).

[0071] Figure 6 A depth pixel Z receiving a useful light signal IR in the near-infrared region is shown, corresponding to the reflection of a parasitic light signal BG and a modulated signal provided by illumination source 16, such as the infrared background light of the scene. Capacitor C illustrates the capacitance at the sensing node SN of the pixel having charge collected by photodiode PH at the end of the sample collection phase. A portion of the charge, CIR, corresponds to the charge generated due to the useful light signal IR, and another portion, CBG, corresponds to the charge generated due to the parasitic light signal BG; these are stored in capacitor C. A routine readout operation then involves reading a signal S representing all the charges stored in capacitor C.

[0072] Figure 7Two depth pixels, ZA and ZB, are shown, each receiving a useful light signal IR and a parasitic light signal BG. At the end of the phase where pixel ZA collects a first sample, the charge present at the sensing node of pixel ZA includes a portion CIRA corresponding to the charge generated by the useful light signal IR and another portion CBGA corresponding to the charge generated by the parasitic light signal BG. At the end of the phase where pixel ZB collects a second sample, the charge present at the sensing node of pixel ZB includes a portion CIRB corresponding to the charge generated by the useful light signal IR and another portion CBGB corresponding to the charge generated by the parasitic light signal BG. Since the parasitic signals are substantially constant and uniform, the contributions CBGA and CBGB are close. According to an embodiment, a capacitor Cd is provided coupled to the two sensing nodes. The readout operation then includes reading the voltage U across the capacitor Cd, which represents the difference between the amounts of charge present at the two sensing nodes, and therefore the difference between the contributions CIRA and CIRB, where the contributions CBGA and CBGB cancel each other out. Interference caused by the parasitic signals is thus suppressed. Furthermore, advantageously, the value U can have a larger dynamic range than the signal S because the unwanted portions of the signal have been suppressed. According to an embodiment, the capacitance of capacitor Cd is in the range of 3 fF to 10 fF.

[0073] Figures 8 to 11 An embodiment of the pixel array arrangement of the acquisition device 12 is shown partially and schematically. Depth sensing units Z are distributed among the 2D image pixels. As an example, in the case of a color image, the 2D image pixels may include pixels R suitable for capturing red light, pixels B suitable for capturing blue light, and pixels G suitable for capturing green light. As an example, 2D image pixels R, G, and B are distributed in a Bayer array, except that one of the pixels G in the Bayer array is replaced by a depth sensing unit Z.

[0074] exist Figure 8 and Figure 9 In the embodiment shown, the acquisition of two samples of each image is performed by a photosensitive unit Z at the same depth, wherein the photosensitive unit may have a similar Figure 4 The structure shown is as follows. Then, capacitor Cd is connected between the two sensing nodes SN and SN' of photodiode PH, which is coupled to photosensitive unit Z.

[0075] according to Figure 8In the embodiment shown, four samples C0, C1, C2, and C3 are acquired by the same depth photosensitive unit Z, wherein samples C0 and C2 are acquired during the acquisition of a first depth image, enabling the acquisition of the difference between samples C0 and C2, and samples C1 and C3 are acquired during the acquisition of a second depth image, enabling the acquisition of the difference between samples C1 and C3. Each depth pixel then includes a single depth photosensitive unit.

[0076] according to Figure 9 In the embodiment shown, photosensitive units Z are distributed among photosensitive unit acquisition samples C0 and C2 and photosensitive unit acquisition samples C1 and C3. This enables the difference between samples C0 and C2 and the difference between samples C1 and C3 to be obtained by acquiring a single depth image. Each depth pixel then includes two depth photosensitive units Z.

[0077] exist Figure 10 and Figure 11 In the embodiment shown, differential readout is performed by acquiring two samples using two different depth photosensitive units Z, each depth photosensitive unit Z being capable of having similar... Figure 3 The structure shown is as follows. A capacitor Cd is then connected between the sensing nodes of two depth pixels Z.

[0078] according to Figure 10 In the embodiment shown, four samples C0, C1, C2, and C3 are acquired by two depth sensing units Z. Samples C0 and C2 are acquired during the acquisition of a first depth image, which enables the acquisition of the difference between samples C0 and C2, and samples C1 and C3 are acquired during the acquisition of a second depth image, which enables the acquisition of the difference between samples C1 and C3. Each depth pixel then includes two depth sensing units.

[0079] according to Figure 11 In the embodiment shown, photosensitive units Z are distributed among photosensitive unit acquisition samples C0 and C2 and photosensitive unit acquisition samples C1 and C3. This enables the difference between samples C0 and C2 and the difference between samples C1 and C3 to be obtained by acquiring a single depth image. Each depth pixel then includes four depth photosensitive units Z.

[0080] Figure 10 and Figure 11 The embodiments shown are relative to Figure 8 and Figure 9 The advantages of the embodiments implemented in this paper are that, Figure 10 and Figure 11The embodiments shown have fewer constraints on the formation of the capacitor Cd and the electronic components of the depth pixel because the electronic components of the depth pixel are distributed on two depth photosensitive units, and the capacitor Cd can be formed on one or more 2D image pixels on and / or between the depth photosensitive units.

[0081] Figures 12A to 12B It shows in such as Figure 10 and Figure 11 The following are the sequential steps of an embodiment of the readout method with the arrangement shown. According to this embodiment, the first plate of capacitor Cd is coupled to the photodiode PHA of the first photosensitive unit ZA via switch SWA1 and to a source at a high reference potential Vdd via switch SWA2, and the second plate of capacitor Cd is coupled to the photodiode PHB of the second photosensitive unit ZB via switch SWB1 and to a source at a high reference potential Vdd via switch SWB2. Switches SWA1, SWA2, SWB1, and SWB2 may correspond to MOS transistors.

[0082] According to an embodiment, the reading cycle includes the following steps:

[0083] a) All switches SWA1, SWA2, SWB1, and SWB2 are closed. Capacitor Cd discharges by connecting its two plates to the same potential Vdd. Figure 12A );

[0084] b) Switches SWA2 and SWB1 are turned on, and switches SWA1 and SWB2 are turned off. This causes charge to be transferred from the photodiode PHB of the second photosensitive unit ZB to the second plate of the capacitor Cd. Figure 12B );

[0085] c) Switches SWA1 and SWB2 are turned on, and switches SWA2 and SWB1 are turned off. This causes charge to be transferred from the photodiode PHA of the second photosensitive unit ZA to the first plate of the capacitor Cd. Figure 12C );and

[0086] d) Switches SWA1, SWB1, and SWB2 are turned off, and switch SWA2 is turned on, which allows the voltage across capacitor Cd to be read. Figure 12D ).

[0087] Steps b) and c) are repeated thousands of times to store the charge corresponding to the incident signal. At step d), for pixels of normal depth, reading can be performed by double sampling.

[0088] According to an embodiment, when performing phase shift calculation based on two consecutive depth images, the readout cycle includes implementing the previously described method to obtain the difference C0-C2 of the first depth image and implementing the previously described method to obtain the difference C1-C3 of the second image.

[0089] Figure 13 It shows in such as Figure 8 and Figure 9 The steps of an embodiment of the readout method are shown in the diagram. In this embodiment, the first plate of capacitor Cd is coupled to the photodiode PH of photosensitive unit Z via switch SWA1 and to a source with a high reference potential Vdd via switch SWA2, and the second plate of capacitor Cd is coupled to the photodiode PH of the same photosensitive unit Z via switch SWB1 and to a source with a high reference potential Vdd via switch SWB2. The control of switches SWA1, SWA2, SWB1, and SWB2 can be as previously described regarding... Figures 12A to 12D Perform as described.

[0090] Phase shift of optical signal According to equation Math_3, determined using only three samples C0, C1, and C2, the previous discussion... Figure 13 and Figures 12A to 12D An embodiment of the described readout method can be implemented by: obtaining differences C0-C1 by a first photosensitive unit, obtaining differences C0-C2 by a second photosensitive unit, and obtaining differences C2-C1 by a third photosensitive unit. Acquisition can be performed using a single depth image, wherein the first and second photosensitive units are different, or using two depth images, wherein the first and second photosensitive units can be the same.

[0091] Figure 14 It shows Figures 12A to 12B A circuit diagram showing a more detailed embodiment of the circuit is provided. As shown in the accompanying drawing, the structure of the photosensitive unit ZB is as follows: Figure 3 The only difference shown is that capacitor C is replaced with capacitor Cd. Switch SWB1 corresponds to transistor 44, and switch SWB2 corresponds to transistor 50. The photosensitive unit ZA also has... Figure 3 The structure shown in the figure, in Figure 3 The circuit references are enclosed in single quotes to distinguish them from the circuit references of the photosensitive unit ZB, except that capacitor C is replaced with capacitor Cd and transistors 46' and 48' are absent, and... Figure 14 The following are indicated by dashed lines. Switch SWA1 corresponds to transistor 44' and switch SWA2 corresponds to transistor 50'. The general structure of the two photosensitive units ZA and ZB coupled by capacitor Cd can therefore include: [The following text appears to be incomplete and requires further context for accurate translation.] Figure 3The structure shown has two photosensitive units, which is less than two transistors.

[0092] Figure 15 It shows Figure 13 A circuit diagram of a more detailed embodiment of the circuit shown in the figure. As shown in the figure, the photosensitive unit Z has Figure 4 The structure shown is different except that the two capacitors C are replaced with capacitor Cd. Furthermore, transistors 66 and 68 are absent, and... Figure 15 The following are indicated by dashed lines. Switch SWB1 corresponds to transistor 44, and switch SWB2 corresponds to transistor 50. Switch SWA1 corresponds to transistor 64, and switch SWA2 corresponds to transistor 70. Therefore, Figure 15 The structure of the photosensitive unit may include more than Figure 4 The diagram shows two transistors with minimal structure.

[0093] According to an embodiment, Figure 1 Device 12 includes:

[0094] - A first sensor, formed inside and on top of a first semiconductor substrate (e.g., a single-crystal silicon substrate), the first sensor including a plurality of depth-sensing units and 2D image pixels; and

[0095] - A second sensor is formed inside and on top of a second semiconductor substrate (e.g., a single-crystal silicon substrate), which is placed close to the surface of the first sensor and includes multiple depth-sensing units.

[0096] Various embodiments and variations have been described. Those skilled in the art will understand that certain features of these various embodiments and variations can be combined, and other variations will occur to them. Finally, based on the functional indications given above, actual implementations of the described embodiments and variations are within the capabilities of those skilled in the art.

Claims

1. A device (12) for acquiring depth images of a scene (Sc), comprising: Depth sensor (Z), and Each capacitor (Cd) includes a first plate and a second plate, wherein each depth photosensitive unit includes: a single photodiode (PH) capable of detecting a reflected light signal corresponding to the reflection of a near-infrared incident amplitude-modulated light signal on the scene, and at least one sensing node (SN, SN') coupled to the photodiode via a single transistor (44, 64), and each capacitor is connected between the sensing nodes of two photosensitive units or between two sensing nodes of the same photosensitive unit. The first depth sensing unit in the depth sensing unit is configured as follows: At least one first sample of charge is provided to the first plate of each of the first capacitors, the at least one first sample of charge being photogenerated by detecting the reflected light signal during a first time period, each of which is equal to a first portion of the modulation period of the incident light signal, and A second sample of charge is provided to the second plate of each first capacitor, the second sample of charge being photogenerated by detecting the reflected light signal during a second time period, each of which is equal to a second portion of the modulation period of the incident light signal, the second time period being offset by a first constant phase shift relative to the first time period, and The second depth sensing unit in the depth sensing unit is configured as follows: At least one third sample of charge is provided to the first plate of each of the second capacitors in the capacitors, the at least one third sample of charge being photogenerated by detecting the reflected light signal during a third time period, each of which is equal to a third portion of the modulation period of the incident light signal, the third time period being offset from the first time period by a second constant phase shift different from the first constant phase shift.

2. The device of claim 1, wherein the first sample and the second sample are provided by the same depth photosensitive unit (Z).

3. The device of claim 1, wherein the first sample and the second sample are provided by photosensitive units (Z) at different depths.

4. The device according to claim 1, wherein The fourth depth sensing unit (Z) of the depth sensing unit is configured to provide a fourth sample of charge to the second plate of each third capacitor, the fourth sample of charge being photogenerated by detecting the reflected light signal during a fourth time period, each of which is equal to a fourth portion of the modulation period of the incident light signal, the fourth time period being offset relative to the first time period by a third constant phase shift different from the first constant phase shift and the second constant phase shift.

5. The device of claim 4, wherein the third sample and the fourth sample are provided by the same depth photosensitive unit.

6. The device of claim 4, wherein the first sample and the second sample are provided by the same depth sensing unit, and wherein the third sample and the fourth sample are provided by a different depth sensing unit than the depth sensing unit that provided the first sample and the second sample.

7. The apparatus of claim 5, wherein the first sample and the second sample are provided by the same photosensitive unit during the acquisition of the first depth image, and wherein the third sample and the fourth sample are provided by the same depth photosensitive unit that provided the first sample and the second sample during the acquisition of the second depth image.

8. The device of claim 4, wherein the third sample and the fourth sample are provided by photosensitive units at different depths.

9. The device of claim 8, wherein the first sample and the second sample are provided by different depth sensing units, and wherein the third sample and the fourth sample are provided by different depth sensing units than the depth sensing units that provided the first sample and the second sample.

10. The apparatus of claim 8, wherein the first sample and the second sample are provided by different depth photosensitive units during the acquisition of the first depth image, and wherein the third sample and the fourth sample are provided by the depth photosensitive unit that provided the first sample and the second sample during the acquisition of the second depth image.

11. The device of claim 1, comprising 2D image pixels for acquiring a 2D image of the scene (Sc).

12. A system (10) for acquiring depth images, comprising: The device (12) according to any one of claims 1 to 11; The light source (16) is configured to emit near-infrared amplitude-modulated incident light signals; as well as The processor (20) is configured to determine the phase shift between the incident light signal and the reflected light signal from the first sample, the second sample and the third sample.

13. The system of claim 12, wherein The fourth depth sensing unit (Z) of the depth sensing unit is configured to provide a fourth sample of charge to the second plate of each third capacitor, the fourth sample of charge being photogenerated by detecting the reflected light signal during a fourth time period, each of which is equal to a fourth portion of the modulation period of the incident light signal, the fourth time period being offset relative to the first time period by a third constant phase shift different from the first constant phase shift and the second constant phase shift, wherein the processor (20) is configured to determine the phase shift between the incident light signal and the reflected light signal from the first sample, the second sample, the third sample and the fourth sample.

Images