Depth scanning image sensor

Abstract

An imaging device according to the present disclosure includes: a pixel array including a plurality of pixels; a signal generator for generating an exposure timing scheme defining an exposure duration of pixels grouped in at least a subset of the pixels, where the exposure duration covers at least a portion between a frame start time and a frame end time of a predefined frame; a synchronizer for synchronizing the generation of the light pulse with the exposure timing scheme; and means for reading out the charge accumulated in the grouped pixels.

Description

Technical Field

[0001] The present disclosure relates to a novel sensor device for 2D (two-dimensional) imaging and 3D (three-dimensional) imaging. The present disclosure also relates to a method for 2D imaging and 3D imaging. The present disclosure also relates to a device for scanning depth and a method for scanning depth. Background Art

[0002] 3D imaging has been around for decades. Increased demand for imaging solutions and technological developments have led to a wide range of applications.

[0003] 3D imaging can be used in different fields. For example, depth sensing is becoming increasingly important in the automotive industry for applications such as preventing car accidents, monitoring the physical condition of car drivers, or autonomous driving. Such applications require essentially real-time responses to be effective.

[0004] In medical diagnosis and surgery, delicate and rare surgeries have become possible due to the ability to reproduce the structure and position of internal organs based on 3D imaging, which allows surgeons to train for surgeries in advance. For this application, slow and precise scanning systems can be used to generate 3D images, as real-time operation is not required.

[0005] In the field of security and data protection, face recognition combined with fingerprint recognition can provide a way to protect people's personal data from being stolen. Here, too, relatively slow systems can be used for 3D imaging, because the main problem is to create a detailed 3D image of an object, such as a face and / or fingerprint, and real-time response is not a problem. Structured light systems and multi-camera systems are widely used for this purpose.

[0006] Several imaging technologies are built on depth information. To obtain depth information, various depth detection solutions have been developed. Depending on the application in question, some of these solutions may be more suitable than others.

[0007] Stereoscopic vision is a known 3D imaging technology that obtains the 3D geometric information of an object from multiple images based on the principle of human binocular vision. Two images of an object are acquired simultaneously or continuously from different perspectives, and the 3D geometric information of the object is derived from these two images based on trigonometry. From this, the 3D contour and position of the object can be calculated. The stereoscopic vision method is a mature 3D imaging technology that produces satisfactory results. However, when the baseline distance is large, the stereoscopic vision method is computationally intensive and its performance is poor. In active stereoscopic vision, structured light is also used. The stereoscopic vision method can be enhanced by using a structured light source (active stereoscopic vision). As mentioned earlier, the stereoscopic vision method can use two sensors and is an early technology for producing 3D images (for example, see Fujifilm FinePix Real 3D).

[0008] In structured light technology, a three-dimensional object is illuminated by projecting a light pattern onto its surface. An example of a light pattern is a parallel band of light. Due to the three-dimensional structure of the object, the lighting pattern looks different on the surface of the three-dimensional object than on a flat surface, i.e., the lighting pattern is reflected differently from the three-dimensional object than from a flat surface. From how the projected light pattern looks in the image, a reconstruction of the three-dimensional structure of the object can be achieved. Structured light technology can use a light source consisting of points with a known geometry. Structured light technology is used in, for example, the iPhone 8.

[0009] Another known technique is time-of-flight (TOF) imaging, where light with a known time profile is shone onto a three-dimensional object and reflected to a camera. The time it takes for the light to reach the camera depends on the distance from the reflection point on the object's surface to the camera. Knowing this time enables the three-dimensional structure of the object to be inferred. The light can be modulated. By sampling the reflected light at a high frequency, a camera or sensor can measure the phase of the reflected light relative to the emitted light. The phase is related to the distance of the point on the object where the light was reflected. This approach is called indirect TOF or i-TOF.

[0010] Direct TOF (dTOF) has recently gained importance in 3D imaging methods due to the improvement of avalanche detectors, silicon photomultipliers (SiPMs), avalanche photodiodes (APDs) or single photon avalanche diodes (SPADs). It is well known that avalanche detectors can generate a large voltage signal when they capture a single electron. This voltage signal can then be used to accurately measure the arrival time of a photon. By the equation

[0011]

[0012] Where c is the speed of light, Δt is the interval between the moment the light pulse is generated and the moment the reflected pulse is detected on the sensor, and the distance d of the object can be calculated.

[0013] SPADs are already available based on CMOS technology, which enables the fabrication of large SPAD arrays. The performance of CMOS-based SPADs still does not meet the performance level of SPADs made with optimized manufacturing techniques, but improvements are being made and enabling the creation of small arrays.

[0014] The technology of SPAD is described in, for example, International Patent Application No. WO2021 / 067377A1 (Sense Photonics / University of Edinburgh). The patent application discloses a light detection and ranging (LIDAR) system, comprising: an emitter array comprising a plurality of emitter units operable to emit a light signal; a detector array comprising a plurality of detector pixels (e.g., SPADs) operable to detect light of a corresponding strobe window between pulses of the light signal; and one or more control circuits.

[0015] However, SPADs have disadvantages. The fill factor, which is the ratio of the photosensitive area of ​​the SPAD to the total area of ​​the SPAD, can be limited. SPADs need to use surface area on the chip to make the drive circuits work. Another disadvantage is the low quantum efficiency, especially in the near-infrared (NIR) region, where dTOF operates for eye protection reasons and to improve the signal-to-noise ratio (SNR) by reducing background light. In addition, SPADs have relatively high power consumption.

[0016] International patent application WO2008 / 138543A1 describes a technique called temporal pixel multiplexing (TPM). Bub et al. (see, e.g., Nature Methods, Vol. 7, pp. 209-211 (2010)) implemented TPM using a digital micromirror device (DMD) and a CCD image sensor. High-speed imaging can be achieved by grouping pixels into subsets of pixels with continuous exposure times to generate frames per subset of pixels. This produces a set of time-separated low-resolution images. The low-resolution images can then be viewed sequentially as a movie, or they can be combined into a single high-resolution image.

[0017] In a further development, International Patent Application No. WO2012 / 013918A1 discloses a pixel design for a CMOS sensor that allows selection of individual pixels. The TPM can be implemented electronically on chip using the disclosed CMOS sensor without using any electromechanical moving parts. A pixel circuit is described that has a shutter signal, a column select signal and a row select signal as inputs.

[0018] International patent application No. WO2019 / 239128A1 discloses a pixel design in which a column sample signal and a row sample signal can be provided to each pixel to read an exposure signal into a storage node of the pixel. The storage node is arranged so that both the column sample signal and the row sample signal need to be in an active state for transmission. The disclosed pixel design does not require a shutter signal.

[0019] U.S. patent application No. US2015 / 319422A1 (Fereyre) US2015319422A1 discloses a method for generating a three-dimensional scene image including information about the distance of each point in the scene, the method using a pulsed light source and an image sensor including pixels capable of accumulating charge generated by light in respective storage nodes of each pixel.

[0020] The present disclosure aims to use a new architecture of 2D sensors to directly measure TOF. Summary of the invention

[0021] The present disclosure relates to a depth sensing technology in which pixels of a 2D image sensor are driven at respective timings to obtain time information about the arrival of photons that generate an image. In addition, the present disclosure relates to a depth scanning image sensor (DSIS) that collects depth information by scanning in a depth direction.

[0022] The image sensor includes a plurality of pixels, and the pixels are controlled by control signals.

[0023] If the pixel operation is controlled in a temporal manner in relation to the exposure timing of individual pixels in the pixel, e.g. synchronized with the emission of, for example, a light pulse, the pixel operation can be used to collect information about the TOF of the light pulse, equivalent to a dTOF measurement. The collected TOF information can be used to derive depth information.

[0024] To simplify the implementation of exposure timing, pixels can be grouped into subgroups so that fewer exposure timings need to be implemented for each image.

[0025] Subsequent images may have different exposure timing schemes assigned to the subgroups of pixels. Different ones of the exposure timing schemes enable scanning within different time ranges.

[0026] In general, in order to collect depth information within a depth range, a series of images with different exposure timing schemes are acquired. Assume that for a specific application, M different exposure timings are required. These M different exposure timings will be applied to each pixel, which will require M different images. However, 3D images usually do not require the same resolution level as the corresponding 2D images, so it is possible to trade off between 2D spatial resolution and the number of images required to reconstruct depth information. By grouping pixels into subgroups of N pixels and applying M different exposure timings to each group of pixels, M / N images (or frames) will be required to scan the depth information. The 2D resolution will be reduced by 1 / N times accordingly.

[0027] Parameters such as light pulse duration, light pulse intensity, light pulse repetition rate, and sensor timing within non-overlapping sub-blocks of the light sensing array are controlled by signals using different timings. The exposure timing is used together with the light pulses to calculate the distance of objects in the scene through time-of-flight (ToF) measurements as well as standard 2D imaging.

[0028] An imaging device for 3D imaging according to the present disclosure includes a pixel array, the pixel array including a plurality of pixels. The plurality of pixels include a plurality of pixel subgroups, and at least one of the plurality of pixel subgroups includes a depth scan pixel (20). The imaging device also includes: a signal generator for generating an exposure timing scheme, the exposure timing scheme defining an exposure duration of the at least one pixel subgroup, wherein some of the exposure durations at least cover a portion between a frame start time and a frame end time of a predefined frame; a synchronizer for synchronizing the generation of light pulses with the exposure timing scheme; and a readout circuit for reading out the charge accumulated in the at least one pixel subgroup.

[0029] The imaging device may further comprise a light source for generating light pulses.

[0030] At least one pixel subgroup may include a photodiode, a transistor for resetting the photodiode, a floating diffusion, a transistor TX for transferring charge from the photodiode to the floating diffusion, a transistor for resetting the floating diffusion, a source follower for converting the charge at the floating diffusion into a voltage, and a selection transistor for providing the voltage to an output line.

[0031] The imaging device may include a storage node connected to an input transistor.

[0032] At least some of the exposure durations may cover different portions between a frame start time and a frame end time.

[0033] The imaging device may also include a processor for processing the accumulated charge after readout.

[0034] The processor may be configured to determine the temporal position of a peak of the reflected light pulse.

[0035] The imaging device may further include a 2D / 3D logic switch for switching between the 2D imaging mode and the 3D imaging mode.

[0036] The signal generator may include a shift register composed of a series of building blocks, the building blocks being connected in series, and some of the building blocks including flip-flops.

[0037] The shift register further includes two signal generators which are connected in series to both ends of the series of base blocks, wherein the two signal generators include flip-flops.

[0038] A method for 3D imaging of an object using a pixel array is also disclosed. As previously described, the pixel array includes a plurality of pixel subgroups, and at least one of the plurality of pixel subgroups includes depth scan pixels. The method includes: generating a light pulse; exposing at least one pixel subgroup of the pixel array grouped into at least one pixel subgroup to a reflected light pulse reflected from an object according to an exposure timing scheme defining an exposure duration of the at least one pixel subgroup, wherein some of the exposure durations at least cover a portion between a frame start time and a frame end time of a predefined frame; reading out the charge accumulated in the at least one pixel subgroup; and determining the temporal position of a peak of the reflected light pulse.

[0039] The method may further include defining at least some of the exposure durations to cover different portions between the frame start time and the frame end time.

[0040] The method may further include repeating the exposing before reading out the charge.

[0041] The method may further include repeating the exposing according to a modified exposure timing scheme and subsequently reading out the charge.

[0042] Determining the temporal position of the peak of the reflected light pulse includes using a linear centroid formula.

[0043] Determining the temporal position of the peak of the reflected light pulse involves the use of a non-linear formula.

[0044] The nonlinear formula may be a nonlinear centroid formula.

[0045] One or both of the frame start time and the frame end time of the frame can be defined based on one or more of at least one wavelength of the light pulse, the power of the light pulse, the duration of the light pulse, the repetition rate of the light pulse, the resolution to be achieved, noise and background light. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1A pixel array is shown that is divided into non-overlapping blocks of depth scan pixels having 2x3 pixels.

[0047] Figure 2 Shown for exposure Figure 1 Exposure timing scheme for 2x3 pixels in one of the DS pixels.

[0048] Figure 3 shows the repetition at different starting points relative to the time course of the light pulse. Figure 2 The exposure timing scheme.

[0049] Figure 4 The determination of the peak position of the reflected light pulse is explained.

[0050] Figure 5 A variable η of a reflected light pulse having a Gaussian shape is described, which is used to shift the peak position of the reflected light pulse.

[0051] Figure 6 A system according to the present disclosure is shown.

[0052] Figure 7 The difference between the linear (calculated) peak position and the nonlinear (actual) peak position for shifting the peak position of the reflected light pulse is shown.

[0053] Figure 8 The resolution achieved by different peak detection algorithms for different exposure time steps is illustrated.

[0054] Fig. 9 The temporal resolution of the nonlinear centroid method for different exposure time intervals is shown.

[0055] Fig.10 An embodiment of a pixel architecture is shown.

[0056] Fig.11 The signal timing for readout of DS pixels of a pixel array for 3D imaging is shown.

[0057] Fig.12 Shows Fig.11 A modified version of the timing shown in .

[0058] Fig.13 The signal timing for readout of consecutive rows of a pixel array for 2D imaging is shown.

[0059] Fig.14 A block diagram showing the architecture of an imaging device.

[0060] Fig.15 An example of the timing of the output waveform of the state machine operation is shown.

[0061] Fig.16 A block diagram of a synchronizer for synchronizing the generation of light pulses with the control of a pixel array is shown.

[0062] Fig.17 A detection mode using an imaging device according to one aspect is shown.

[0063] Fig.18 A signal generator is shown. DETAILED DESCRIPTION

[0064] The present invention will now be described based on the accompanying drawings. It should be understood that the embodiments and aspects of the present invention described herein are only examples and do not limit the scope of protection of the claims in any way. The present invention is defined by the claims and their equivalents. It should be understood that the features of one aspect or embodiment of the present invention may be combined with the features of different aspects and / or embodiments of the present invention.

[0065] The present disclosure relates to an imaging device 100 (see Fig.14 The present disclosure relates to an imaging device 100 (e.g., an imaging device in a semiconductor device) and semiconductor sensor-based technologies, and more specifically to the creation of 3D images. More specifically, the present disclosure relates to a method for depth sensing and depth scanning using a semiconductor imaging device 100. In one aspect, the semiconductor imaging device 100 is a CMOS imaging device.

[0066] Let us consider a semiconductor imaging device 100 having a pixel array 10 having X×Y pixels 20. The pixels 20 may be of the same type and constructed in the same manner. Figure 1 In the example of FIG. 1 , 9×6 pixels 20 are depicted in the pixel array 10. However, it is understood that the pixel array 10 may include a different number of pixels 10 than is conventional in the art. The pixel array 10 is configured to detect the reflected light pulses 50 generated to illuminate the object to be imaged 105 (see FIG. 1 ). Figure 6 ).

[0067] The light pulses 50 are generated by a light source 107. The light source 107 is connected to a laser driver 150 and receives control signals from the laser driver 150 related to the generation of the light pulses 50. The light pulses 50 reflected by the object 105 to be imaged are detected by the pixel array 10 by exposing the pixels 20 to the reflected light pulses 50. In one aspect of the present disclosure, the imaging device 100 includes the light source 107. In another aspect, the imaging device 100 may include a light directing device for directing the light pulses 50 along a selected direction to the object 105 to be imaged or a portion thereof.

[0068] The light source 107 has characteristics such as: at least one wavelength (which includes at least one central wavelength or dominant wavelength); narrow band; peak power; light pulse duration; pulse repetition rate; and average power. These characteristics can be selected using the laser driver 150. In one aspect of the present disclosure, the semiconductor imaging device 100 is constructed on a silicon substrate, and at least one wavelength of the light source 107 can be selected to be between about 0.8 μm and about 1.05 μm. In another aspect, in the imaging device 100, the range of at least one wavelength can extend to about 1.55 μm or 1.6 μm. In other words, the range of at least one wavelength extends from about 0.8 μm to 1.6 μm. Therefore, the light pulse 50 can be composed of infrared light (IR). In order to improve the detection of infrared radiation, in one aspect of the present disclosure, the material of the substrate can be selected to have a higher detection efficiency, such as germanium. Using IR light to generate the light pulse 50 can enable human interaction without perceiving the light pulse 50 because the human eye is less sensitive to IR light. In addition, solar radiation at the ground may result in the inability to detect light pulses 50 having at least one wavelength in the visible light frequency range. Due to atmospheric absorption, there is a decrease in the IR range of the solar radiation spectrum at the ground. When light in the IR range is used to generate light pulses 50, the level of background noise from solar radiation can be reduced. The settings of the average power of the light pulses 50, the peak power of the light pulses 50, the duration of the light pulses 50, and the repetition rate of the light pulses 50 can be interdependent, and can also be set in consideration of the object to be imaged 105 and / or imaging conditions (such as distance range, resolution, and eye safety). In one aspect of the present disclosure, the light pulses 50 have a relatively short duration, which has a relatively high peak power and a repetition rate in the order of kHz. In addition, the field of view (FOV) of the light source 107 can match the FOV of the imaging device 100. The light pulses 50 can illuminate the entire FOV of the imaging device 100, thereby eliminating the need for a light guide.

[0069] The X×Y pixels 20 are grouped into one or more subgroups 40 in the pixel array 10. One or more subgroups 40 are referred to as depth scan pixels or DS pixels. Assume that M different timings are required to obtain sufficient depth information about the object to be imaged 105. The number M depends on the depth range (or distance) to be detected (or depth range) and the resolution selected for the imaging application. Assume that the depth range is identified by its boundaries d1 and d2, that is, d1≤d≤d2. The imaging device 100 will be in a time range relative to the moment when the light pulse 50 leaves the light source 107 and travels towards the object to be detected 105 Some of the reflected light pulses 50 arriving at the imaging device 100 (i.e., arriving at the pixel array 10) are detected within the imager. In order to detect a single reflected light pulse 50, repeated exposures of the imaging device 100 can be performed according to the shape of the reflected light pulse 50 (e.g., the time interval spanned by the reflected light pulse 50) and the exposure timing scheme 45 (explained below). However, it is conceivable that in addition to a single light pulse 50 that repeatedly illuminates the object 105 to be imaged, other light pulses 50 can also be used to illuminate the object 105 to be imaged. The other light pulses 50 can have, for example, different shapes. According to the exposure timing scheme 45 and the previously defined time range, the imaging device 100 will perform a certain number (M) of exposures with different exposure timings, as will be further explained below.

[0070] The number of DS pixels 40 in the pixel array 10 may be different from the number of pixels 20 in the pixel array 10, for example, less than or equal to the number of pixels 20 in the pixel array 10. If the pixels 20 are divided into a plurality of DS pixels 40, for example, A×B DS pixels 40, where A≤X; B≤Y, then some of the DS pixels 40 have (X×Y) / (A×B) pixels 20. We define n=X / A and m=Y / B. The resulting DS resolution of the pixel array is n×m.

[0071] In the above example, the pixel array 10 is divided into DS pixels 40 regularly, that is, the DS pixels 40 have the same number of pixels 20. In one aspect of the present disclosure, the X×Y pixels 20 of the pixel array 10 may be divided into DS pixels 40 in an irregular manner, where n and m vary across the pixel array 10. In this aspect, the pixel array 10 includes DS pixels 40 of different sizes, thereby producing images with varying DS resolutions.

[0072] The subgroup 40 (or DS pixel 40) has n×m pixels 20. Figure 1 In the example, n=3 and m=2. Figure 2 An example of an exposure timing scheme 45 for a pixel 20 in one of the DS pixels 40 is shown. The exposure timing scheme 45 defines the exposure duration of the i-th pixel (also referred to as the i-th pixel) in the subgroup 40. For the i-th pixel of the DS pixel 40, i=1, ..., n×m (in Figure 2 In the example of , n×m=6), the exposure (the incident light from the reflected pulse 50 is integrated over the duration of the exposure) is typically from time t start_exp (i) Start, and at time t end_exp (i) End.

[0073] exist Figure 2 In the example shown, for i=1, ..., 6, t end_exp (i) = tend_exp In other words, the six pixels 20 of the DS pixel 40 are end_exp ) stops being exposed to the reflected light pulse 50. Therefore, the i-th pixel in this example has the value given by the formula t exp (i) = t end_exp -t start_exp (i) Given the exposure duration t exp (i). In this example, the exposure duration t exp The length of (i) decreases as the pixel index i increases. However, it should be noted that Figure 2 The exposure scheme 45 shown in FIG. 4 is merely an example of an exposure timing scheme 45 .

[0074] Other exposure timing schemes 45 may be employed. In one aspect, the exposure of the i-th pixel (i=1, ..., 6) may be started at a common time, and the time at which the exposure of the i-th pixel ends may be distributed or dispersed. For example, the exposure duration of the first pixel i=1 may be stopped first, and the exposure duration of the sixth pixel i=6 may be stopped last.

[0075] On the other hand, the exposure end time of the i-th pixel may correspond to the exposure start time of another pixel 10 in the DS pixels 40 , such as but not limited to the i+1-th pixel.

[0076] Once the exposure timing scheme 45 is selected, the DS pixel 40 can be calculated for the i-th exposure duration t exp The amount of light received during (i) (i = 1, ..., 6). Figure 1 In the example shown, the i-th pixel (i=1, . . . , 6) corresponding to the DS pixel 40 is obtained at the i-th exposure duration t exp (i), (i = 1, ..., 6) during the integrated light intensity, see Figure 2 The integrated light amount, for example, at the ith exposure duration t exp The number of photons or amount of light energy received during (i) (i=1, . . . , 6) can be inferred from the amount of charge accumulated by the pixel, either directly or based on the difference between adjacent pixels in the pixel 20 in the DS pixel 40 .

[0077] Typically, this method generates n×m exposure durations. When switching from the X×Y pixel array 10 to the A×B DS pixel array (eg Figure 1 The number n×m also indicates a reduction in spatial resolution when the number of images is denoted as shown in FIG. 1 . The method according to the present disclosure enables a reduction in the number of images to be obtained for a 3D point map. The reduction in the number of images, as described above, is M / (m×n).

[0078] To obtain a higher number of exposure durations, DS pixels 40 may be exposed to subsequent reflected light pulses 50 during a subsequent exposure after object 105 was exposed during a previous exposure. In one aspect, during the subsequent exposure, exposure timing scheme 45 may be the same as exposure timing scheme 45 used during the previous exposure. In another aspect, during the subsequent exposure, exposure timing scheme 45 may be different than exposure timing scheme 45 used during the previous exposure.

[0079] Exposing the DS pixel 40 to a preceding light pulse 50 and subsequently to a succeeding light pulse 50 enables detection of the reflected light pulses 50 over a longer period of time without further reducing the spatial resolution of the pixel array 10. Depending on the size of the pixel array 10, i.e., the number of pixels 20 in the pixel array 10, there is a trade-off between the number of pixels 20 per DS pixel 40 (i.e., the reduction in spatial resolution) and the number of illuminations required to obtain sufficient depth information of the object 105 to be imaged.

[0080] exist Figure 3 In the aspects of the present disclosure shown, the repetition Figure 2 The exposure timing scheme 452 is shown. In the repeated exposure using the exposure timing scheme 45, the start time t of the exposure timing scheme 45 relative to the timing of the light pulse 50 delay (k) Continuous delays. Figure 2 The exposure timing scheme 45 shown is repeatedly applied to the i-th pixel in the DS pixel 40 (or a subset 40 of pixels 20 ), i=1, . . . , 6.

[0081] Repeated exposure enables a longer time range to be covered, for example, a time range covering at least the time course of the generated light pulse 50 and / or at least the time course of the reflected light pulse 50, such as Figure 3 This time range is Figure 3 It is called DSIS frame. Figure 3 In the example shown, the illumination of the object to be imaged 105 is repeated N times. The subsequent repetitions cover slightly time-shifted parts of the time course of the light pulse 50. At the end of each DSIS frame and before moving to a different repetition scheme, the sensor needs to be read out.

[0082] In some cases, the readout of the sensor takes longer than acquiring a single DSIS frame. For example, for measurements at a range of 10m, the maximum time of flight is 66.7ns. State-of-the-art readout rates are around 25 billion pixels per second. Therefore, a megapixel sensor can be read at 40,000 frames per second, which corresponds to a readout cycle of 25μs>>66.7ns. Most of the power of the sensor is consumed during the readout process. The power consumed depends on the detailed architecture of the sensor and can be in the order of several watts for very high-speed sensors with many pixels. As explained above, the power of the light pulses 50 needs to be kept low to limit the risk of danger. Furthermore, the operating system 200 (see Figure 6 ), the system comprising a light source 107 and an imaging device 100. Therefore, it is convenient to repeat the DSIS frame several times without reading the sensor, which is possible in a pixel architecture that allows charge accumulation. In this case, more charge is accumulated in the i-th pixel by repeatedly irradiating the object to be imaged 105. If the generated light pulse 50 and the exposure timing scheme 45 remain unchanged during the repetition, the charge accumulated in the i-th pixel grows proportionally to the number of repetitions.

[0083] Repeated exposure increases the signal-to-noise ratio (SNR). Increasing the SNR is advantageous, for example, when the available power of the light source 107 is low. In other cases, the output power of the light source 107 must be limited for eye safety reasons, and such limitation of the output power may result in poor quality results. Repeated illumination of the object 105 with light pulses 50 enables overcoming the limited amount of light incident on the pixel array 10 by increasing the SNR.

[0084] The time range covered by the repeated irradiation may depend on the distance d at which the object 105 to be imaged is arranged from the pixel array 10 and / or the light source 107 (or the two-dimensional distribution range of the distance d when scanning the object 105 in two dimensions). As explained above, the formula is related to the distance d between the light source 107 and / or the pixel array 10 and the time Δt required for the light pulse to travel twice the distance d. Therefore, if the distance of the object 105 is 1m, Δt=6.7ns; if the distance of the object 105 is 100m, Δt=670ns. Therefore, repeating the measurement 1000 times will result in a coverage time range of 6.7μs for a distance of 1m and a coverage time range of 0.67s for a distance of 100m. Therefore, the accumulation of charge in the pixel 10 by repeated illumination can be achieved relatively quickly. In addition, the charge accumulation reduces the time spent reading the pixel 10, which, as explained above, is relatively time-consuming.

[0085] In one aspect, some of the pixels 10 have an architecture in which a charge storage device is arranged within the pixel architecture. Charge representing the amount of light falling on the pixel is transferred to the charge storage device. The imaging device 100 may also include a control device for controlling the transfer of charge to the charge storage device.

[0086] On the other hand, the charge storage device may be arranged outside the pixel 20 , for example, at a periphery of the pixel array 10 .

[0087] An example of a method of imaging the object to be imaged 105 will now be described. However, variations of the described algorithm are possible.

[0088] The method according to the present disclosure may be improved by the fact that the light pulses may last longer than the cumulative exposure duration of the exposure timing scheme 45 .

[0089] The method uses interpolation between pixels to achieve TOF resolution that is superior to the exposure time interval used during exposure of the pixel array 10. The present disclosure relates to a method that may be referred to as nonlinear center of gravity (COG). Nonlinear COG represents an improvement over linear COG in that it can handle any arbitrary shape of light pulse 50, whereas linear COG only works well for rectangular light pulses 50. The nonlinear COG method is used to determine the temporal position of the peak of the reflected pulse 50. Based on the determined peak position, depth information about the object to be imaged 105 is measured, as explained below.

[0090] exist Figure 4 , a reflected pulse 50 reaching the pixel array 10 is depicted. Figure 4 The start time t of the exposure timing scheme 45 is shown. delay (k) After the i-th pixel (i=1, ..., n×m) of the DS pixel 40 is exposed, for example, using Figure 2 The exemplary exposure timing scheme 45 shown in FIG. 4 performs subtraction of the integrated charge Q(j) of the neighboring pixels (i.e., the j+1th pixel and the jth pixel) in the pixel 20 within the DS pixel 40, i.e., at the exposure time t exp (j) The charge accumulated during the period and corresponding to the amount of incident light from the reflected light pulse 50.

[0091] In this way, a series of integrated charges Q(j) is obtained, j = 0, ..., n × m-1. This series of charges Q(j) corresponds to Figure 4 The box in the figure has a value equal to ΔT(j)=t exp (j+1)-t exp In one aspect, the series of charges Q(j) can also be obtained by dividing the exposure time by the exposure time step (or exposure time interval) of the time length of (j). exp(j) is defined to be obtained directly without overlap, in which case the subtraction step is omitted. In Figure 2 In the example of exp (j) overlap.

[0092] This series of charges Q(j) can be regarded as representing the time course of the charges accumulated in the DS pixel 40 within the exposure time step ΔT(j) (j = 0, …, n×m−1) relative to the starting time t delay (k). In this case, the exposure time step ΔT(j) can be regarded as a “time point”.

[0093] In Figure 2 the case shown, all exposure time steps ΔT(j) have the same length, i.e., ΔT(j)=ΔT. However, generally the length of the exposure time step ΔT(j) varies.

[0094] More generally, considering the starting time t delay (k) of the exposure timing scheme 45, a set of charges Q(j,k) corresponding to the exposure time interval ΔT(j,k) is obtained. As Figure 4 shown, the exposure time steps are arranged such that there is no overlap between the exposure time steps ΔT(j,k). In addition, when comparing two exposure time steps ΔT(j,k) and ΔT(l,m), if k < m (regardless of the values of j and l) or if k = m and j < l, then the exposure time step ΔT(j,k) precedes the exposure time step ΔT(l,m).

[0095] More generally, it is possible for the exposure time steps ΔT(j,k) to overlap. The meaning that the exposure time step ΔT(j,k) precedes the exposure time step ΔT(l,m) is that at least some time points within the exposure time step (or exposure time interval) ΔT(j,k) occur earlier than the time points within the exposure time step ΔT(l,m). For simplicity, the case where the exposure time steps ΔT(j,k) overlap is not analyzed below.

[0096] When the pixel array 10 is repeatedly exposed to the reflected light pulse 50 using a series of starting times t delay (k), the series of charges Q(j,k) can be regarded as representing the time course of the charges accumulated in the DS pixel 40 within the exposure time (where j = 0, …, m×n−1). If the exposure time step (or interval) is constant, these exposure times become t(j,k)=t delay (k)+j·ΔT. In this aspect of the present disclosure, the exposure time t(j,k) is defined relative to the start of the exposure time step ΔT(j,k).

[0097] exist Figure 4 In another aspect shown in FIG. , the center of the exposure time step (or exposure time interval) ΔT(j,k) can be used as the exposure time defined as In this aspect, if the exposure time step (or interval) is constant, the exposure time becomes

[0098] Let the index z=1,2,… denote the index pair j and k of the time-ordered series ΔT(j,k). In other words, (i,j)→z, and if z>z' for all z=1,2,…, then ΔT(z)>ΔT(z'). The mapping also defines a corresponding series of accumulated charges Q(z).

[0099] In order to find the time position of the reflected light pulse 50, for example, the time position T0 of the peak value (see Figure 4 ), run the search algorithm on the series Q(z).

[0100] For example, linear filtering can be applied in a first step by convolving the series of accumulated charges Q(z) with a known filter with a time expression h(z) to improve the SNR of the measured values. In a selection step, those elements Q(z0), Q(z1), ... of the series Q(z) that are above a certain threshold are selected. i )'s adjacent elements (z i ±1)、Q(z i ±2)..., further processing the element Q(z i ), i = 1, 2, .... Based on the determined adjacent elements Q(z i ±1)、Q(z i ±2)…, the existence of the reflected light pulse 50, such as the peak value of the reflected light pulse 50, can be detected in the detection step.

[0101] In the case of peak detection, the corresponding element of the series of charges Q(z) can be expressed as Q(z m ). Based only on the element Q(z m ) and element Q(z m′ )=max{Q(z m -1),Q(z m +1)}, that is, two adjacent elements Q(z m ±1) has a larger charge value, and the time position T0 of the peak is determined in the step of determining the peak position (see Figure 4 ).

[0102] In one aspect, the step of determining the peak position comprises calculating the time position T0 of the peak based on the COG method, i.e., using the linear COG formula T0=(Q(z m )·t(z m )+Q(z m′ )·t(z m′ )) / (Q(z m )+Q(z m′ )).

[0103] In the case where the exposure time step (or exposure time interval) is constant, that is, ΔT(j,k) = ΔT, and the center of the exposure time step is used as the reference time point, the linear COG formula can be simplified. Define Q left Q(z m ) and Q(z m′ ) with a smaller index (which is denoted as p), define Q right Q(z m ) and Q(z m′ ) with the larger index (which is denoted as p+1). With these definitions, the linear COG formula yields By defining η = Q right / (Q left +Q right ), the result can be written as

[0104] If the reference time The above results can be simplified by rewriting them. In this case, T0 = ΔT(p+η).

[0105] If the time course of the reflected light pulse 50 is completely contained within one exposure time interval (or exposure time step), then Q(z m′ ) is zero, which leads to η = 1 (if z m′ <z m ) or η=0 (if z m′ >z m ). If the peak of the reflected light pulse 50 is located on the boundary between adjacent exposure time intervals in the exposure time interval, then due to the symmetry of the Gaussian form of the reflected light pulse 50 (which leads to η=0.5), Q(z m )=Q(z m′ )(within statistical uncertainty / noise fluctuations).

[0106] In embodiments of the method that deviate from the above, the variable η takes different values ​​depending on the relative time length of the exposure time interval and the time course of the reflected light pulse 50. Specifically, as the size of the exposure time interval decreases relative to the time course of the reflected light pulse 50, the range of variation of the defined parameter η also decreases accordingly. This means that when the time course of the reflected light pulse 50 is completely contained within one exposure time interval, the above-mentioned situation [0≤η≤1] constitutes an upper limit on the size (or length) of the range in which the parameter η can vary.

[0107] exist Figure 5 In FIG. 5 , the calculated variable η (y-axis) is shown for the range (x-axis) of the time position T0 of the peak value of the reflected light pulse 50 between the center position of a given exposure time interval and the center position of a subsequent exposure time interval ΔT. The reflected light pulse 50 has a Gaussian form with σ=5ns. Therefore, the time course of the reflected light pulse 50 is wider than the selected exposure time interval ΔT=9ns. Figure 5 As shown, when the peak position T0 is shifted from the center position of the exposure time interval by increments of ΔT / 1000 within an interval of length ΔT, the variable η is found to be almost linear. The variable η varies between about 0.2 and about 0.8 (this range is smaller than the range of 0≤η≤1 discussed previously).

[0108] Depending on the relative time lengths of the exposure time interval and the time course of the reflected light pulse 50, a known correction will be applied to the linear relationship. Figure 7 The calculated peak position T determined based on the above linear relationship is shown. 0_calculated The difference between the actual peak position T0 and the peak position T0 of different shifts, as in Figure 5 The calculation results are shown.

[0109] Linear COG formula based on peak position Therefore, the linear relationship between η and the relative peak position T′0 is η=T′0 / ΔT.

[0110] In the more general nonlinear case, the relationship between η and the relative peak position T′0 is η=F(T′0 / ΔT). By inverting the function F, we obtain T′0 / ΔT=F -1 (η). Since η is measured and F is known, it can be calculated The function F can be determined by calibrating the imaging device 100. For calibration purposes, the time course of the light pulses 50 can be predetermined. Furthermore, for calibration purposes, the structure of the object 105 to be imaged can be predetermined.

[0111] In the step of determining the depth (or distance), based on the TOF formula The measured time position T0 is converted into depth (or distance).

[0112] For illustration purposes, in the statistical simulation, the function F is numerically calculated based on an ideal reflected light pulse 50 without noise. During the statistical simulation, noise is repeatedly added and the distribution of the peak position T0 is determined. Figure 8 The corresponding results, i.e. the standard deviation of the distributions determined by the different calculation methods, are plotted in Figure 2, which illustrates the effect of the nonlinear COG correction compared to other calculation methods used in the statistical simulations. It can be seen that the nonlinear COG interpolation significantly reduces the error in the time measurement.

[0113] To get a more detailed understanding of the achieved temporal resolution, Fig. 9 Only the standard deviation of the temporal position T0 of the peak obtained using the nonlinear COG method is shown. is calculated as a function of the exposure time step (or exposure time interval) of the reflected light pulse 50 with a fixed width of 5 ns. Even for a relatively large exposure time step (or exposure time interval) ΔT j =ΔT=15ns, the nonlinear COG calculation can also achieve a time resolution of the order of 100ps, which is less than one percent (1 / 100) of the exposure time step (or exposure time interval).

[0114] In the example described above, two charge values ​​Q(z) from the series Q(z) are used in the nonlinear COG method. m ) and Q(z m′ ). This should not be considered a requirement. More samples from the series Q(z) can be used to improve the determination of the temporal position T0 of the peak. For example, two or more charge values ​​to the right and left of the peak can be used and the temporal position (x-axis) of the peak of the reflected light pulse 50 can be calculated based on the adjusted nonlinear correction formula.

[0115] The present disclosure relates to the architecture of the pixel 20 of the pixel array 10. In one aspect of the present disclosure, the pixel 20 is a 5T pixel (see Fig.10), includes five transistors, such as CMOS transistors. Transistor AB is used to reset the photodiode PD node. Transistor AB can also provide anti-blooming. Transistor TX is used to transfer the integrated charge from the photodiode PD to the floating diffusion FD node, which must be reset before the transfer occurs. The resetting of the floating diffusion FD is accomplished by turning on the reset transistor RST. Once the charge reaches the floating diffusion FD, the charge is converted into a voltage by the source follower. The source follower includes a source follower input transistor SFI and a current source connected to the output line OUT and located at a peripheral device of the pixel array 10. The voltage is provided to the output line OUT via the source follower input transistor SFI and the selection transistor SEL. The voltage can be detected on the output line OUT through the source follower input transistor SFI and the selection transistor SEL acting as a switch.

[0116] The pixel 20 provides correlated double sampling (CDS), which allows for the elimination of undesired offsets when measuring voltage and / or current. CDS further enables noise reduction, for example, thermal noise, such as kTC noise.

[0117] The pixel 20 further provides a global shutter mode and / or a rolling shutter mode. In the global shutter mode, all pixels 20 in the pixel array 10 have equal exposure timing and exposure time interval. 2D imaging can be performed in the rolling shutter mode, thereby improving spatial resolution.

[0118] In one aspect of the present disclosure, 3D imaging may be performed in a DSIS global shutter operation. Figure 2 An example of DSIS global shutter operation is shown in the exposure timing diagram of FIG. In the DSIS global shutter mode, the i-th pixel in a number (e.g., all) of the DS pixels 40 of the pixel array 10 has an equal exposure timing and exposure time interval. In other words, in the DSIS global shutter mode, it is equivalent to performing a global shutter operation on a subarray of the pixel array 10, the subarray including the i-th pixel of the DS pixel 40, where i=1, ...6. Fig.17 As shown, a series of sub-arrays are exposed successively.

[0119] Other possible embodiments may differ at the schematic level or in layout, with different implementations at the process level. At the schematic level, a storage node may be added after the source follower input transistor SFI of the source follower. The storage node allows the results of the disclosed method to be stored and read out when further implementations of the method are initiated.

[0120] At the pixel layout and manufacturing level, the photodiode doping profile and shape can be designed to speed up charge collection. For infrared detection using infrared light, that is, the wavelength range of the light pulse 50 and / or the reflected light pulse 50 is about 0.8um to 1.6um, which is the preferred wavelength range for 3D imaging or 3D measurement, and the absorption length is generally greater than the absorption length of visible light. The larger absorption length causes at least some photons of the light pulse 50 to penetrate deeper into the substrate of the photodiode PD. The deeper penetration causes the charge generated by the light pulse 50 to travel a greater distance to the collection node. Faster charge collection enables the pixel 20 to operate faster, which is beneficial for 3D imaging.

[0121] 3D imaging can be performed in the following manner. Transistor RST performs a reset of the floating diffusion FD at the beginning of a frame. Thereafter, transistor AB and transistor TX implement a timing, such as an exposure timing scheme 45.

[0122] exist Fig.11 In the Figure 1 4 is an example of a timing sequence for 2×3 DS pixels 40 in order to realize an exposure timing scheme 45 for 3D imaging of an object 105. However, the timing sequence can be similarly applied to n×m DS pixels 40. Fig.11 It can be seen that a frame (referred to as "frame 0") includes a GRST (global reset) phase, a TRIG (trigger) phase, a DSIS (depth scan image sensor) phase, and a RO (readout) rolling shutter phase. This frame can be an element in a series of frames.

[0123] During the GRST phase, the floating diffusion FD of the i-th pixel (i=1, ..., 6) is reset by turning on the RST transistor. In addition, the photodiode PD of the i-th pixel (i=1, ..., 6) is released without any residual collected charge and is held at its pinned voltage by transistor AB.

[0124] During the TRIG phase, a trigger is generated internally or externally to initiate the DSIS phase.

[0125] After receiving the trigger, during the DSIS phase, the DSIS synchronizer 110 generates a start pulse signal for the DSIS signal generator 120 to generate the exposure timing scheme 45. The generation of the start pulse for the DSIS signal generator 120 takes into account the start time t that has been previously set. delay (k) The DSIS signal generator 120 generates signal timing for the transistor AB and the transistor TX of the i-th pixel (i=1, ..., 6) to reproduce the exposure timing scheme 45, for example Figure 2 The exposure timing scheme 45 is shown.

[0126] from Fig.11It can be seen that during the DSIS phase, after the DSIS synchronizer 110 generates a start pulse signal, as the value of the index i in the example shown increases, the AB signal of the i-th pixel (i=1, ..., 6) becomes low level in a continuous manner, for example.

[0127] In addition, during the DSIS phase, when the DSIS synchronizer 110 generates a start pulse signal, the TX signal becomes high. The moment when the TX signal becomes high depends on the operation mode. In mode 0, the TX signal becomes high shortly after the start pulse signal and before the AB signal of the i-th pixel (i=1, ..., 6) becomes low. In mode 1, the TX signal becomes high after the AB signal of the i-th pixel (i=1, ..., 6) becomes low.

[0128] During the DSIS phase, the DSIS synchronizer 110 generates a stop pulse signal, which is sent to the DSIS signal generator 120 to end the exposure. The stop pulse signal causes the TX signal to become low and the charge is transferred from the photodiode PD node to the floating diffusion FD. The AB signal of the i-th pixel (i=1, ..., 6) becomes high, which resets the photodiode PD node.

[0129] In the RO rolling phase, the i-th pixel (i=1, ..., 6) is read in rolling shutter mode. In rolling shutter mode, rows can be read in pairs.

[0130] In one aspect of the present disclosure, a CDS (correlated double sampling) phase may be included in the timing after the GRST phase and before the TRIG phase. The CDS phase includes reading a reset level that is different between pixels 20 in the pixel array 10 and depends on: (i) the voltage applied to the gate of the RST transistor; (ii) VDD; (iii) charge injection and feedthrough on the FD node; (iv) reset noise (which is a characteristic of the CMOS imaging device 100 and is due to the fact that the reset operation effectively samples the voltage on the photodiode node PD); (iv) dark current (which is typically negligible in pixel designs involving the use of a pinned photodiode PD).

[0131] Fig.12 An aspect of the present disclosure is shown, which enables accumulation of charge on the floating diffusion FD node. As described above, the accumulation of charge enables improvement of SNR. By modifying Fig.11 The timing of the DS pixel 40 is used to realize the accumulation of charge. Fig.11 The floating diffusion FD node is reset at the beginning of the frame during the GRST phase. The TRIG phase and the DSIS phase follow the GRST phase, as shown in Figure 2. Fig.11This is repeated before reading out the i-th pixel (i=1,…,6) in the RO rolling shutter phase, as shown in the timing of Fig.12 shown.

[0132] exist Fig.12 In the aspect shown, within a single frame, several light pulses 50 are generated, and the charge photogenerated by the several reflected light pulses 50 in the diode is transferred to the floating diffusion FD through the transistor TX after each reflected light pulse 50 during the DSIS phase. This process is repeated n times, so that the charge stored in the floating diffusion FD node is the result of the accumulation of the photogenerated charges of the n generated light pulses 50. Fig.12 In the RO rolling phase, the DSIS phase is accordingly referred to as the DSIS-with-accumulation phase. Once the DSIS-with-accumulation phase is completed, the readout scheme is performed as before during the RO rolling phase.

[0133] 2D imaging can be performed in the following exemplary manner, which implements a rolling shutter mode, where both light integration (or charge integration) and pixel readout are controlled row by row. 2D imaging includes CDS, an example of which is Fig.13 For consecutive rows in the pixel array 10, Fig.13 The timing is shown in the example of 13. For several rows, the SEL signal (or SEL transistor control signal) is kept high during at least a portion of the RST RO (reset readout) phase and the SIGNAL RO (signal readout) phase, which allows the reset and signal level values ​​to be read and CDS to be performed.

[0134] During the SINT (start integration) phase, the address counter 140 is activated and starts generating addresses to be sent to the row decoder 145. The row decoder 145 decodes the addresses and generates a row pointer signal, which is sent to the row driver to start integrating some of the pixels 20 belonging to the selected consecutive rows of the pixel array 10, for example, Fig.13 In the example of , rows 0 and 1 are integrated first, followed by rows 2 and 3. The AB signal of some of the pixels 20 belonging to the selected row is first set high and then set low to start the exposure. This AB signal ensures that the diode is empty, i.e., the residual collected charge is cleared before starting another exposure. Since the collected charge is not completely transferred to the floating diffusion node during the readout phase, the residual collected charge may be from the previous exposure. By activating the AB transistor, all the charge accumulated in the previous exposure is cleared, allowing another exposure to start cleanly.

[0135] During the RST RO (reset readout) phase, when the RST signal of some of the pixels 20 belonging to the selected row becomes high level during the RST RO phase and then becomes low level again, resetting is performed on the floating diffusion FD of some of the pixels 20 belonging to the selected row. When the SEL signal becomes high level for some of the pixels 20 belonging to the selected row during the RST RO phase and becomes low level during the SIGNAL RO (signal readout) phase, the voltage at the floating diffusion node FD is provided to the column output line OUT for readout due to the accumulated charge.

[0136] After the reset value is read, for some of the pixels 20 belonging to the selected row, the accumulated or integrated charge is transferred from the photodiode PD to the floating diffusion FD of the selected row by applying a pulse to the transistor TX of the selected row. Subsequently, the accumulated or integrated charge is transferred to the output line OUT for reading, while for some of the pixels 20 belonging to the selected row, the SEL signal continues to be high.

[0137] After the selected row has been read out, the foregoing stages will be applied to further selected consecutive rows of the pixel array 10. The selection and readout of rows of the pixel array 10 may be repeated until the pixel array 10 has been completely read. Once the sensor has been completely read, the process may restart from the initially selected row.

[0138] refer to Fig.13 The signal timings described are only possible signal timings for a rolling shutter readout mode. Other timings may also be applied, such as those known to those skilled in the art.

[0139] Fig.14 A block diagram of an imaging device 100 according to the present disclosure is shown.

[0140] The imaging device 100 includes a DSIS synchronizer 110. The DSIS synchronizer 110 synchronizes the laser driver 150 with the exposure timing scheme 45. The DSIS synchronizer 110 is programmable to take into account the start time t of the exposure timing scheme 45. delay (k).

[0141] Fig.16 The DSIS synchronizer 110 is shown in more detail. An externally or internally generated trigger signal (also shown in Fig.14The 3D frame or 3D imaging frame is started by the counter 1510. After being synchronized with the internal clock through the signal clk_in, the counter 1510 starts counting. Three registers are used to store delay values: the first register 1520 is used to store the light pulse generation delay (Laser_start), the second register 1530 is used to store the exposure start delay (DSIS_start), and the third register 1540 is used to store the exposure end delay (DSIS_stop). The first register 1520, the second register 1530 and the third register 1540 are all coupled to the comparator.

[0142] The comparator coupled to the first register 1520 for storing the light pulse generation delay generates a Laser_start pulse and delivers the Laser_start pulse to the laser driver 150 when the counter reaches the value of the light pulse generation delay. The comparator coupled to the second register 1530 for storing the exposure start delay generates a DSIS_start pulse and delivers the DSIS_start pulse to the basic state machine to turn on the signal generation of the pixel array 10 when the counter reaches the value of the exposure start delay. The comparator coupled to the third register 1540 for storing the exposure end delay generates a DSIS_stop pulse and delivers the DSIS_stop pulse to the basic state machine to turn off the signal generation of the pixel array 10 when the counter reaches the value of the exposure end delay.

[0143] The state machine includes a multiplexer and a D flip-flop. The operation of the state machine is illustrated by the timing of the output waveform, as shown in Fig.15 The output of the state machine is a signal called DSIS_start_stop which is used as input in the DSIS signal generator block 120 .

[0144] The imaging device 100 further includes a DSIS signal generator 120, such as Fig.14 As shown. The DSIS signal generator 120 generates a signal that is fed to the signal distribution network and ultimately reaches the pixel driver of the pixel 20 of the pixel array 10. The timing and delay of the signal for the pixel array 10 are determined and controlled. In one aspect of the present disclosure, the signal distribution network includes one or more clock trees. The signal distribution network may include an associated clock tree for each pixel signal (e.g., AB signal and TX signal).

[0145] Fig.17 The DSIS signal generator 120 is shown. The DSIS signal generator 120 includes several (e.g., Fig.16 In the example shown, there are six) blocks 121, which correspond to the number of 2×3 pixels 20 in each DS pixel 40 (see Figure 1). Several building blocks 121 are connected in series through several AND gates 122. The number of building blocks 121 deployed in the DSIS signal generator 120 can be any number selected for a particular imaging application. In one aspect, the number of building blocks 121 deployed in the DSIS signal generator 120 can be programmable. Programmability allows flexibility in the design of imaging applications. In this regard, the structure of the pixel array 10 can include xy-addressable pixels 20 (for example, as disclosed in WO2019 / 239128).

[0146] The DSIS_start_stop_AB generator 123 receives the DSIS_start_stop signal from the DSIS synchronizer 110. The DSIS_start_stop_AB generator 123 generates the DSIS_start_stop_AB signal, which is received by a plurality of AND gates 122 connected to a plurality of base blocks 121 of the DSIS signal generator 120. The DSIS_start_stop_AB signal is the result of an inverted and delayed signal of the DSIS_start_stop signal from the DSIS synchronizer 110, such as Fig.18 shown.

[0147] The DSIS signal generator 120 has an asynchronous reset async_rst. As described above, the building blocks 121 are connected in series to form a signal generator shift register. The signal generator shift register generates an output signal for the i-th pixel (i=1, ..., 6) by inverting the output of the flip-flop (e.g., D flip-flop) of the building block, called AB , i=1, ..., 6. As described above, the number of pixels 20 per DS pixel 40 is not limited to 2×3=6, but is generally n×m. Fig.18 As shown, the input of the base block's flip-flop is the result of the logical AND between the DSIS_start_stop_AB signal and the output of the previous flip-flop, as Fig.18 As shown (output AB in series <1> The input of the first flip-flop is associated with a high logic value).

[0148] The DSIS_start_stop_AB signal is used as a synchronous reset for the DSIS signal generator 120. The DSIS_start_stop_AB signal is received by a number of AND gates 122 connected to a number of building blocks 121, and sets the output of the DSIS signal generator 120 to a known state when the DSIS_start_stop_AB signal is low. The output of the DSIS signal generator 120 is the inverted output of each flip-flop, which means that: (i) When DSIS_start_stop_AB is low, the outputs all remain at a high voltage level. (ii) When DSIS_start_stop_AB is high, the output of the flip-flop of a building block depends on the output of the flip-flop of the previous building block. Point (ii) describes the stage when the AB signal becomes low, as described with respect to Fig.15 As explained, an exposure timing scheme 45 is generated.

[0149] In fact, when DSIS_start_stop_AB is high, once the first rising edge of the clock is hit, the output of the first flip-flop goes low. When the second rising edge comes, the output of the second flip-flop (which depends on the output of the previous flip-flop) goes low. The output then propagates to the subsequent flip-flops until the end of the series is reached. At this point, all outputs remain low until the DSIS_start_stop_AB signal goes low again. This event causes all outputs to go high simultaneously once the first rising edge of the clock is hit.

[0150] Regarding the TX signal, two operation modes are allowed: In mode 0, the TX signal becomes high from the beginning before the first AB signal becomes low. In mode 1, the TX signal becomes high after the last AB signal becomes low.

[0151] AB and TX timings are strictly related to each other. In one aspect, the TX signal generator generates the TX signal. In this aspect, the TX signal generator is integrated into the shift register previously introduced. For example, two blocks 127, 128 including a flip-flop 127 and an AND gate 128 connected to the flip-flop input are added (attached) to the series connection of the base block 121 of the DSIS signal generator 120. One of the blocks 127, 128 is arranged in TX_mode <0> The other of blocks 127 and 128 is arranged in TX_mode <1> The other end of the series connection of the base block 121 (see Fig.18 ). The DSIS_start_stop_TX signal is received by an AND gate 128 connected to a corresponding flip-flop 127 and sets the output of the corresponding flip-flop 127 to a known state when the DSIS_start_stop_TX signal is low. Switching between the two modes is ultimately performed by a multiplexer 126, which selects the TX signal according to the operating mode (see Fig.18 ).

[0152] The imaging device 100 further includes a 2D / 3D logic switch 130, such as Fig.14 The 2D / 3D logic switch 130 switches between the 2D imaging mode and the 3D imaging mode.

[0153] The device 100 according to the present disclosure may include a processor for processing the accumulated charge after readout.

Claims

1. An imaging device (100) for 3D imaging, include: A pixel array (10) comprising a plurality of pixels (20); The plurality of pixels (20) include a plurality of pixel (20) subgroups (40), at least one pixel (20) subgroup (40) of the plurality of pixel (20) subgroups (40) includes a depth scan pixel (20); A signal generator (120) for generating an exposure timing scheme (45) defining an exposure duration (t exp (i)), wherein the exposure duration (t exp Some of (i)) cover at least a portion between a frame start time and a frame end time of a predefined frame; a synchronizer (110) for synchronizing the generation of light pulses (50) with the exposure timing scheme (45); as well as A readout circuit is provided for reading out the charge accumulated in the at least one subgroup (40) of pixels (20).

2. The imaging device (100) of claim 1, further comprising a light source (107) for generating the light pulses (50).

3. An imaging device (100) as described in claim 1 or 2, wherein the at least one pixel (20) subgroup (40) includes a photodiode (PD), a transistor (AB) for resetting the photodiode (PD), a floating diffusion (FD), a transistor TX for transferring charge from the photodiode (PD) to the floating diffusion (FD), a transistor (RST) for resetting the floating diffusion (FD), a source follower for converting the charge at the floating diffusion (FD) into a voltage, and a selection transistor (SEL) for providing the voltage to an output line (OUT).

4. The imaging device (100) of one of claims 1 to 3, further comprising a storage node connected to the input transistor SF.

5. The imaging device (100) according to one of claims 1 to 4, wherein the exposure duration (t exp At least some of (i)) cover different parts between the frame start time and the frame end time.

6. The imaging device (100) of one of claims 1 to 5, wherein the imaging device (100) further comprises a processor for processing the accumulated charge after readout.

7. The imaging device (100) of claim 6, wherein the processor is configured to determine the time position (T 0 ).

8. The imaging device (100) of one of claims 1 to 7, further comprising a 2D / 3D logic switch for switching between a 2D imaging mode and a 3D imaging mode.

9. The imaging device (100) of one of claims 1 to 8, wherein the signal generator comprises a shift register consisting of a series of building blocks (121), the building blocks (121) being connected in series and some of the building blocks (121) comprising triggers.

10. The imaging device (100) of claim 9, wherein the shift register further comprises two TX signal generators, the two TX signal generators being connected in series to both ends of the series of base blocks (121), wherein the two TX signal generators comprise triggers.

11. A method for 3D imaging of an object (105) using a pixel array (10), wherein the pixel array (10) comprises a plurality of pixel (20) subgroups (40), and at least one of the plurality of pixel (20) subgroups (40) comprises a depth scan pixel (20), wherein the method include: generating a light pulse (50); The at least one subset (40) of pixels (20) of the pixel array (10) is exposed to the light pulse (50) reflected from the object (105) according to an exposure timing scheme (45), the exposure timing scheme (45) defining an exposure duration (t exp (i)), wherein the exposure duration (t exp Some of (i)) cover at least a portion between a frame start time and a frame end time of a predefined frame; reading out the charge accumulated in the at least one subset (40) of pixels (20); Determine the time position (T 0 ).

12. The method of claim 11, further comprising: include: Define the exposure duration (t exp (i)) to cover different parts between the frame start time and the frame end time.

13. The method according to claim 11 or 12, include: The exposure is repeated before the charge is read out.

14. The method according to one of claims 11 to 13, include: The exposure is repeated according to a modified exposure timing scheme (45) and the charge is then read out.

15. The method as claimed in one of claims 11 to 14, wherein the time position (T 0 ) include: Use the linear centroid formula.

16. The method as claimed in one of claims 11 to 15, wherein the time position (T 0 ) include: Use a nonlinear formula.

17. The method of one of claims 11 to 16, wherein the non-linear formula is a non-linear centroid formula.

18. A method as claimed in one of claims 11 to 17, wherein one or both of the frame start time and the frame end time of the frame are defined based on at least one wavelength of the light pulse (50), the power of the light pulse (50), the duration of the light pulse (50), the repetition rate of the light pulse (50), the resolution to be achieved, one or more of noise and background light.

Images