3d-stacked SPAD sensor with in-pixel multi-frame storage

Abstract

The system discloses a pixel including a memory and a single-photon avalanche diode (SPAD). The pixel is operable in a first mode and a second mode. The system further includes using high density static random-access memory (SRAM) based pixels for multi-bin histogramming for time-of-flight measurements and multi-frame storage intensity imaging.

Description

PATENT Attorney Docket No. 103033-P041 WO1-1539924Client Ref. No. P041WO13D-STACKED SPAD SENSOR WITH IN-PIXEL MULTI-FRAME STORAGECROSS REFERENCE TO RELATED APPLICATIONS

[0001] This PCT application claims priority to U. S. Provisional Application No.63 / 736,523 filed December 19, 2024, the entirety of which is incorporated by reference herein for all purposes.BACKGROUND

[0002] Time-of-flight (ToF) based imaging is used in a number of applications, including range finding, depth profiling, and 3D imaging, for example light imaging, detection, and ranging (LiDAR, or lidar). Direct time-of-flight (dToF) measurement includes directly measuring the length of time between emitting radiation from emitter elements and sensing the radiation by sensor elements after reflection from an object or other target. The distance to the target can be determined from the measured length of time. Indirect time-of-flight measurement includes determining the distance to the target by phase modulating the amplitude of the signals emitted by the emitter elements of the lidar system and measuring phases (e.g., with respect to delay or shift) of the echo signals received at the sensor elements of the lidar system. These phases can be measured with a series of separate measurements or samples.

[0003] In specific applications, the sensing of the reflected radiation in either direct or indirect time-of-flight systems can be performed using an array of detectors, for example an array of Single-Photon Avalanche Diodes (SPADs). One or more detectors can define a sensor for a pixel, where a sensor array can be used to generate a lidar image for the depth (range) to objects for respective pixels.

[0004] When imaging a scene, these sensors, which can also be referred to as ToF sensors or photosensors, can include circuits that time-stamp and count incident photons as reflected from a target. Data rates can be compressed by histogramming timestamps. For instance, for each pixel, a histogram having bins (also referred to as “time bins”) corresponding to different ranges of photon arrival times can be stored in memory, and photon counts can beaccumulated in different time bins of the histogram according to their arrival time. A time bin can correspond to a duration of, e.g., 1 ns, 2 ns, or the like. Some lidar systems can perform in-pixel histogramming of incoming photons using a clock-driven architecture and a limited memory block, which can provide a significant increase in histogramming capacity. However, since memory capacity is limited and typically cannot cover the desired distance range at once, such lidar systems can operate in ‘"strobing” mode. “Strobing” refers to the generation of detector control signals (also referred to herein as “strobe signals” or “strobes”) to control the timing and / or duration of activation (also referred to herein as “detection windows” or “strobe windows”) of one or more detectors of the lidar system, such that photon detection and histogramming is performed sequentially over a set of different time windows, each corresponding to an individual distance subrange, so as to collectively define tire entire distance range. In other words, partial histograms can be acquired for subranges or “time slices” corresponding to different sub-ranges of the distance range and then amalgamated into one full-range histogram. Thousands of time bins (each corresponding to respective photon arrival times) can typically be used to form a histogram sufficient to cover the typical time range of a lidar system (e.g., microseconds) with the typical time-to-digital converter (TDC) resolution (e.g., 50 to 100 picoseconds).

[0005] Reflected light from the emitter elements can be received using a sensor array. Hie sensor array can be an array of SPA Ds for an array of pixels, also referred to as channels, where each pixel includes one or more SPADs to form one or more detector components. These SPADs can work in conjunction with other circuits, for example address generators, accumulation logic, memory circuits, and the like, to generate a lidar image.SUMMARY

[0006] This disclosure relates generally to using a high density static random-access memory (SRAM) based pixels for multi-bin histogramming for time-of-flight measurements and multi -frame storage intensity imaging.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a simplified block diagram of a lidar system according to some embodiments;

[0008] FIG. 2 is a simplified block diagram of components of a time-of-flight measurement system or circuit according to some embodiments;

[0009] FIG. 3 illustrates the operation of a typical lidar system that can be improved by embodiments;

[0010] FIG. 4 shows a histogram according to embodiments of the present invention;

[0011] FIG. 5 shows the accumulation of a histogram over multiple pulse trains for a selected pixel according to embodiments of the present invention;

[0012] FIG. 6 illustrates a pixel utilizing an SRAM according to an embodiment of the present invention;

[0013] FIG. 7 illustrates a timing diagram of the global shutter pixel operation;

[0014] FIG. 8 shows an intensity image captured by the sensor; and

[0015] FIG. 9 demonstrates the sensor in high speed (burst) imaging mode where a 60 kHz flickering LED was imaged at 16 intervals, each stored in a separate memory location.DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS1. Example Lidar System

[0016] FIG. 1 illustrates an example light-based 3D sensor system 100, for example a Light Detection and Ranging (LiDAR, or lidar) system, in accordance with some embodiments of the invention. Lidar system 100 can include a control circuit 110, a timing circuit 120, driver circuitry 125, an emitter array 130 and a sensor array 140. Emitter array 130 can include a plurality of emitter units (or emitter elements) 132 arranged in an array (e.g,, a one- or two-dimensional array) and sensor array 140 can include a plurality of sensors or sensor elements 142 arranged in an array (e.g., a one- or two-dimensional array). The sensors 142 can be depth sensors, for example time-of-flight (ToF) sensors. In some embodiments each sensor 142 can include, for example, one or more single-photon detectors, for example Single¬ Photon Avalanche Diodes (SPADs). In some embodiments, each sensor 142 can be coupled to an in-pixel memory' block 600 (shown in FIG. 6) that accumulates histogram data for that sensor 142, and the combination of a sensor and in-pixel memory' circuitry’ is sometimes referred to as a “pixel” 142. Each emiter unit 132 of the emitter array 130 can include one or more emiter elements that can emit a radiation pulse (e.g., light pulse) or continuous wave signal at a time and frequency controlled by a timing generator or driver circuitry 125. Insome embodiments, the emitter units 132 can be pulsed light sources, for example LEDs or lasers including vertical cavity surface emitting lasers (VCSELs) that emit a cone of light (e.g., infrared light) having a predetermined beam divergence.

[0017] Emitter array 130 can project pulses of radiation into a field of view of the lidar system 100. Some of the emitted radiation can then be reflected back from objects in the field, for example targets 150. The radiation that is reflected back can then be sensed or detected by the sensors 142 within the sensor array 140. Control circuit 110 can implement a processor that measures and / or calculates the distance to targets 150 based on data (e.g., histogram data) provided by sensors 142. In some embodiments control circuit 110 can measure and / or calculate the time of flight of the radiation pulses over the journey from emitter array 130 to target 150 and back to the sensors 142 within the sensor array 140 using direct or indirect time-of-flight (ToF) measurement techniques.

[0018] In some embodiments, emitter array 130 can include an array (e.g., a one- or two-dimensional array) of emitter units 132 w here each emitter unit is a unique semiconductor chip having one or more individual VCSELs (sometimes referred to herein as emitter elements) formed on tire chip. An optical element 134 and a diffuser 136 can be disposed in front of the emitter units such that light projected by the emitter units passes through the optical element 134 (which can include, e.g,, one or more Fresnel lenses) and then through diffuser 136 prior to exiting lidar system 100. In some embodiments, optical element 134 can be an array of lenses or lenslets (in which case the optical element 134 is sometimes referred to herein as “lens array 134” or “lenslet array 134”) that collimate or reduce the angle of divergence of light received at the array and pass the altered light to diffuser 136, The diffuser 136 can be designed to spread light received at the diffuser over an area in the field that can be referred to as the field of view' of the emitter array (or the field of illumination of the emiter array). In general, in these embodiments, emitter array 130, lens array or optical element 134, and diffuser 136 cooperate to spread light from emitter array 130 across the entire field of view' of the emitter array. A variety of emitters and optical components can be used.

[0019] The driver circuitry' 125 can include one or more driver circuits, each of which controls one or more emitter units. The driver circuits can be operated responsive to timing control signals with reference to a master clock and / or power control signals that control the peak power and / or the repetition rate of the light output by the emitter units 132. In some embodiments, each of the emitter units 132 in the emitter array 130 is connected to andcontrolled by a separate circuit in driver circuitry 125. In other embodiments, a group of emitter units 132 in the emitter array 130 (e.g., emitter units 132 in spatial proximity to each other or in a common column of the emitter array), can be connected to a same circuit within driver circuitry 125. Driver circuitry 125 can include one or more driver transistors configured to control the modulation frequency, timing, and / or amplitude of the light (optical emission signals) output from the emitter units 132.[0020| In some embodiments, a single event of emitting light from the multiple emitter units 132 can illuminate an entire image frame (or field of view); this is sometimes referred to as a "‘flash” lidar system, Other embodiments can include non-flash or scanning lidar systems, in which different emitter units 132 emit light pulses at different times, e.g., into different portions of the field of view. The maximum optical power output of the emitter units 132 can be selected to generate a signal-to-noise ratio of the echo signal from the farthest, least reflective target at the brightest background illumination conditions that can be detected in accordance with embodiments described herein. In some embodiments, an optical filter (not shown) for example a bandpass filter can be included in the optical path of the emitter units 132 to control the emitted wavelengths of light.

[0021] Light output from the emitter units 132 can impinge on and be reflected back to lidar system 100 by one or more targets 150 in the field. Tire reflected light can be detected as an optical signal (also referred to herein as a return signal, echo signal, or echo) by one or more of the sensors 142 (e.g., after being collected by receiver optics 146), converted into an electrical signal representation (sometimes referred to herein as a detection signal), and processed (e.g., based on time-of-flight techniques) to define a 3-D point cloud representation 160 of a field of view 148 of the sensor array 140. In some embodiments, operations of lidar systems can be performed by one or more processors or controllers, for example control circuit 110.

[0022] Sensor array 140 includes an array of sensors 142. In some embodiments, each sensor 142 can include one or more photodetectors, e.g., SPADs. And in some particular embodiments, sensor array 140 can be a very’ large array made up of hundreds of thousands or even millions of densely packed SPADs. Receiver optics 146 and receiver electronics (including timing circuit 120) can be coupled to the sensor array 140 to power, enable, and disable all or parts of the sensor array 140 and to provide timing signals thereto. In some embodiments, sensors 142 can be activated or deactivated with at least nanosecond precision (supporting time bins of 1 ns, 2 ns, etc.), and in various embodiments, sensors 142 can be sindividually addressable, addressable by group, and / or globally addressable. The receiver optics 146 can include a bulk optic lens that is configured to collect light from the largest field of view that can be imaged by the lidar system 100, which in some embodiments is determined by the aspect ratio of the sensor array 140 combined with the focal length of the receiver optics 146.

[0023] In some embodiments, the receiver optics 146 can further include various lenses (not shown) to improve the collection efficiency of the sensors and / or an anti-reflective coating (also not shown) to reduce or prevent detection of stray light. In some embodiments, a spectral filter 144 can be positioned in front of the sensor array 140 to pass or allow passage of “signal” light (i.e., light of wavelengths corresponding to wavelengths of the light emitted from the emitter units) but substantially reject or prevent passage of non-signal light (i.e., light of wavelengths different from the wavelengths of the light emitted from the emitter units).

[0024] The sensors 142 of sensor array 140 are connected to the timing circuit 120. The timing circuit 120 can be phase-locked to the driver circuitry 125 of emitter array 130. The sensitivity of each of the sensor elements 142 or of groups of sensors 142 can be controlled. For example, when the sensor elements 142 include reverse-biased photodiodes, avalanche photodiodes (APD), PIN diodes, and / or Geiger-mode avalanche diodes (e.g., SPADs), the reverse bias can be adjusted. In some embodiments, a higher overbias provides higher sensitivity.

[0025] In some embodiments, control circuit 110, which can be, for example, a microcontroller or microprocessor, provides different emitter control signals to the driver circuitry 125 of different emitter units 132 and / or provides different signals (e.g., strobe signals) to the timing circuit 120 of different sensors 142 to enable / disable the different sensors 142 to detect tire echo signal (or returning light) from the target 150. The control circuit 110 can also control memory storage operations for storing data indicated by the detection signals in a non-transitory memory or memory array that is included therein or is distinct therefrom.

[0026] FIG. 2 further illustrates components of a ToF measurement system or circuit 200 in a lidar application in accordance with some embodiments described herein, The circuit 200 can include a processor circuit 210 (for example a digital signal processor (DSP)), a timing generator 220 that controls timing of the illumination source (illustrated by way of examplewith reference to a laser emitter array 230), and an array of sensors (illustrated by way of example with reference to a sensor array 240). The processor circuit 210 can also include a sequencer circuit (not shown in FIG. 2) that is configured to coordinate operation of emitter units within the illumination source (emitter array 230) and sensors within the sensor array 240.

[0027] The processor circuit 210 and the timing generator 220 can implement some of the operations of the control circuit 110 and the driver circuitry 125 of FIG. 1. Similarly, emitter array 230 and sensor array 240 can be representative of emitter array 130 and sensor array 140 in FIG. 1. The laser emitter array 230 can emit laser pulses 235 at times controlled by the timing generator 220. Light 245 from the laser pulses 235 can be reflected back from a target (illustrated by way of example as object 250) and can be sensed by sensor array 240. The processor circuit 210 implements a pixel processor that can measure or calculate the time of flight of each laser pulse 235 and its reflected light 245 over the journey from emitter array 230 to object 250 and back to the sensor array 240.

[0028] Tire processor circuit 210 can provide analog and / or digital implementations of logic circuits that provide the necessary timing signals (for example quenching and gating or strobe signals) to control operation of the single-photon detectors of the sensor array 240 and that process the detection signals output therefrom. For example, individual single-photon detectors of sensor array 240 can be operated such that they generate detection signals in response to incident photons only during the gating intervals or strobe windows that are defined by the strobe signals, while photons that are incident outside the strobe windows have no effect on the outputs of the single-photon detectors. More generally, the processor circuit 210 can include one or more circuits that are configured to generate detector or sensor control signals that control the timing and / or durations of activation of the sensors 142 (or particular single-photon detectors therein), and / or to generate respective emitter control signals that control the output of light from the emitter units 132.

[0029] Detection events can be identified by the processor circuit 210 based on one or more photon counts indicated by the detection signals output from the sensor array 240, which can be stored in a non-transitory memory 215. In some embodiments, the processor circuit 210 can include a correlation circuit or correlator that identifies detection events based on photon counts (referred to herein as correlated photon counts) from two or more single-photon detectors within a predefined window (time bin) of time relative to one another, referred to herein as a correlation window or correlation time, where the detection signals indicatearrival times of incident photons within the correlation window. Since photons corresponding to the optical signals output from tire emitter array 230 (also referred to as signal photons) can arrive relatively close in time with each other, as compared to photons corresponding to ambient light (also referred to as background photons), the correlator can be configured to distinguish signal photons based on respective times of arrival being within the correlation time relative to one another. Such correlators and strobe windows are described, for example, in U. S. Patent Application Publication No. 2019 / 0250257, entitled “Methods and Systems for High-Resolution Long Range Flash Lidar,” which is incorporated by reference herein in its entirety for all puiposes.

[0030] The processor circuit 210 can be small enough to allow for three-dimensionally stacked implementations, e.g., with the sensor array 240 “stacked” on top of processor circuit 210 (and other related circuits) that is sized to fit within an area or footprint of the sensor array 240. For example, some embodiments can implement the sensor array 240 on a first substrate, and transistor arrays of the processor circuit 210 on a second substrate, with the first and second substrates / wafers bonded in a stacked arrangement, as described for example in U. S. Patent Application Publication No. 2020 / 0135776, entitled “High Quantum Efficiency Geiger-Mode Avalanche Diodes Including High Sensitivity Photon Mixing Structures and Arrays Thereof,” the disclosure of which is incorporated by reference herein in its entirety for all puiposes.

[0031] The pixel processor implemented by the processor circuit 210 can be configured to calculate an estimate of the average ToF aggregated over hundreds or thousands of laser pulses 235 and photon returns in reflected light 245. The processor circuit 210 can be configured to count incident photons in the reflected light 245 to identify detection events (e.g., based on one or more SPADs within the sensor array 240 that have been “triggered”) over a laser cycle (or portion thereof).

[0032] The timings and durations of the detection windows can be controlled by a strobe signal (Strobe#i or Strobe). Many repetitions of Strobe#i can be aggregated (e.g., in the pixel) to define a sub-frame for Strobe#!, with subframes i = 1 to n defining an image frame. Each sub-frame for Strobe#! can correspond to a respective distance sub-range of the overall imaging distance range. In a single-strobe system, a sub-frame for Strobe#! can correspond to tire overall imaging distance range and is the same as an image frame since there is a single strobe. Tire time between emiter unit pulses (which defines a laser cycle, or more generally emitter pulse frequency) can be selected to define or can otherwise correspond to the desiredoverall imaging distance range for the ToF measurement circuit 200. Accordingly, some embodiments described herein can utilize range strobing to activate and deactivate sensors for durations or ‘‘detection windows” of time over the laser cycle, at variable delays with respect to the firing of the laser, thus capturing reflected correlated signal photons corresponding to specific distance sub-ranges at each window / frame, e.g., to limit tire number of ambient photons acquired in each laser cycle.[00331 The strobing can turn off and on individual photodetectors or groups of photodetectors (e.g., for a pixel), e.g., to save energy during time intervals outside the detection window. For instance, a SPAD or other photodetector can be turned off during idle time, for example after an integration burst of time bins and before a next laser cycle. As another example, SPADs can also be turned off while all or part of a histogram is being read out from non-transitory memory 215. Yet another example is when a counter for a particular time bin reaches the maximum value (also referred to as “bin saturation”) for the allocated bits in the histogram stored in non-transitory memory 215. A control circuit can provide a strobe signal to activate a first subset of the sensors while leaving a second subset of the sensors inactive. In addition or alternatively, circuitry' associated with a sensor can also be turned off and on as specified times.2. Detection of Reflected Pulses[00341 The sensors be arranged in a variety of ways for detecting reflected pulses. For example, the sensors can be arranged in an array, and each sensor can include an array of photodetectors (e.g., SPADs). A signal from a photodetector indicates when a photon was detected and potentially how many photons were detected. For example, a SPAD can be a semiconductor photodiode operated with a reverse bias voltage that generates an electric field of a sufficient magnitude that a single charge earner introduced into the depletion layer of the device can cause a self-sustaining avalanche via impact ionization. The initiating charge carrier can be photo-electrically generated by a single incident photon striking the high field region. The avalanche is quenched by a quench circuit, either actively (e.g., by reducing the bias voltage) or passively (e.g., by using the voltage drop across a serially connected resistor), to allow the device to be “reset” to detect other photons. This single-photon detection mode of operation is often referred to as “Geiger Mode,” and an avalanche can produce a current pulse that results in a photon being counted. Other photodetectors can produce an analog signal (in real time) proportional to the number of photons detected. The signals fromindividual photodetectors can be combined to provide a signal from the sensor, which can be a digital signal. This signal can be used to generate histograms.2.1. Time-of-Flight Measurements and Detectors

[0035] FIG, 3 illustrates the operation of a typical lidar system that can be improved by some embodiments. A laser or other emitter (e.g., within emitter array 230 or emitter array 130) generates a light pulse 310 of short duration. The horizontal axis represents time and the vertical axis represents power. An example laser pulse duration, characterized by the full-width half maximum (FWHM), is a few nanoseconds, with the peak power of a single emitter being around a few watts. Embodiments that use side emitter lasers or fiber lasers can have much higher peak powers, while embodiments with small diameter VCSELs could have peak powers in the tens of milliwatts to hundreds of milliwatts.[0036| A start time 315 for the emission of the pulse does not need to coincide w ith the leading edge of the pulse. As shown, the leading edge of light pulse 310 can be after the start time 315. One can want the leading edge to differ in situations where different patterns of pulses are transmitted at different times, e.g., for coded pulses. In this example, a single pulse of light is emitted. In some embodiments, a sequence of multiple pulses can be emitted, and the term “pulse train"’ as used herein refers to either a single pulse or a sequence of pulses.

[0037] An optical receiver system (which can include, e.g., sensor array 240 or sensor array 140) can start detecting received light at the same time as the laser is started, i.e., at the start time. In other embodiments, the optical receiver system can start at a later time, which is at a known time after the start time for tire pulse. The optical receiver system detects background light 330 initially and after some time detects the laser pulse reflection 320. The optical receiver system can compare the detected light intensity against a threshold to identify the laser pulse reflection 320. Where a sequence of pulses is emitted, the optical receiver system can detect each pulse. The threshold can distinguish the background light 330 from light corresponding to the laser pulse reflection 320.

[0038] The time-of-flight 340 is the time difference between tire pulse 310 being emitted and the pulse reflection 320 being received. The time difference can be measured by subtracting the emission time of the pulse 310 (e.g., as measured relative to the start time) from a received time of the pulse reflection 320 (e.g., also measured relative to the start time). The distance to tire target can be determined as half the product of the time-of-flight and thespeed of light. Pulses from the laser device reflect from objects in the scene at different times, depending on start time and distance to the object, and the sensor array detects the pulses of reflected light.2.2. Histogram Signals from Photodetectors

[0039] One mode of operation of a lidar system is time-correlated single photon counting (TCSPC), which is based on counting single photons in a periodic signal. This technique works well for low levels of periodic radiation which is suitable in a lidar system. This time correlated counting can be controlled by a periodic signal, e.g., from timing generator 220.

[0040] The frequency of the periodic signal can specify a time resolution within which data values of a signal are measured. For example, one measured value can be obtained for each photosensor per cycle of the periodic signal. In some embodiments, the measurement value can be the number of photodetectors that triggered during that cycle. The time period of the periodic signal corresponds to a time bin, with each cycle being a different time bin.

[0041] FIG. 4 shows a histogram 400 according to some embodiments described herein. The horizontal axis corresponds to time bins as measured relative to start time 415. As described above, start time 415 can correspond to a start time for an emitted pulse train. Any offsets between rising edges of the first pulse of a pulse train and the start time for either or both of a pulse train and a detection time interval can be accounted for when determining the received time to be used for the time-of-flight measurement. In this example, the sensor pixel includes a number of SPADs, and the vertical axis corresponds to the number of triggered SPADs for each time bin. Other types of photodetectors can also be used. For instance, in embodiments where APDs are used as photodetectors, the vertical axis can correspond to an output of an analog-to-digital converter (ADC) that receives the analog signal from an APD. It is noted that APDs and SPADS can both exhibit saturation effects. Where SPADs are used, a saturation effect can lead to dead time for the pixel (e.g., when all SPADs in the pixel are immediately triggered and no SPADs can respond to later-arriving photons). Where APDs are used, saturation can result in a constant maximum signal rather than the dead-time based effects of SPADs. Some effects can occur for both SPADs and APDs, e.g., pulse smearing of very oblique surfaces can occur for both SPADs and APDs.

[0042] The counts of triggered SPADs for each of the time bins correspond to the different bars in histogram 400. The counts at the early time bins are relatively low and correspond to background noise 430. At some point, a reflected pulse 420 is detected, The correspondingcounts are much larger and can be above a threshold that discriminates between background and a detected pulse, The reflected pulse 420 results in increased counts in four time bins, which might result from a laser pulse of a similar width, e.g., a 4 ns pulse when time bins are each 1 ns.

[0043] The temporal location of the time bins corresponding to reflected pulse 420 can be used to determine the received time, e.g., relative to start time 415. In some embodiments, matched filters can be used to identify a pulse pattern, thereby effectively increasing the signal-to-noise ratio and allowing a more accurate determination of the received time. In some embodiments, the accuracy of determining a received time can be less than the time resolution of a single time bin. For instance, for a time bin of 1 ns, a resolution of one time bin would correspond to a distance about 15 cm. However, it can be desirable to have an accuracy of only a few centimeters.

[0044] Accordingly, a detected photon can result in a particular time bin of the histogram being incremented based on its time of arrival relative to a start signal, e.g., as indicated by start time 415, The start signal can be periodic such that multiple pulse trains are sent during a measurement. Each start signal can be synchronized to a laser pulse train, with multiple start signals causing multiple pulse trains to be transmitted over multiple laser cycles (also sometimes referred to as “shots”). Thus, a time bin (e.g., from 200 to 201 ns after the start signal) would occur for each detection interval. The histogram can accumulate the counts, with the count of a particular time bin corresponding to a sum of the measured data values all occurring in that particular time bin across multiple shots. When the detected photons are histogrammed based on such a technique, the result can be a return signal having a signal to noise ratio greater than that from a single pulse train by the square root of the number of shots taken.

[0045] FIG. 5 shows the accumulation of a histogram over multiple pulse trains for a selected pixel according to some embodiments described herein. FIG. 5 shows three detected pulse trains 510, 520 and 530. Each detected pulse train corresponds to a transmitted pulse train that has a same patern of two pulses separated by a same amount of time. Thus, each detected pulse train has a same pulse pattern, as shown by two time bins having an appreciable value. Counts for other time bins are not shown for simplicity of illustration, although the other time bins can have non-zero values (generally lower than the values in time bins corresponding to detected pulses).

[0046] In the first detected pulse train 510, the counts for time bins 512 and 514 are the same. This can result from a same (or approximately the same) number of photodetectors detecting a photon during each of the two time bins, or approximately the same number of photons being detected during the two time bins, depending on the particular photodetectors used. In other embodiments, more than one consecutive time bin can have a non-zero value; but for ease of illustration, individual nonzero time bins have been shown.

[0047] Time bins 512 and 514 respectively occur 458 ns and 478 ns after start time 515. The displayed counters for the other detected pulse trains occur at the same time bins relative to their respective start times. In this example, start time 1 is identified as occurring at time 0, but the actual time is arbitrary. The first detection interval for the first detected pulse train can be 1 μs. Thus, the number of time bins measured from start time 515 can be 1,000. After, this first detection interval ends, a new pulse train can be transmitted and detected. The start and end of the different time bins can be controlled by a clock signal, which can be part circuitry that acts as a time-to-digital converter (TDC).

[0048] For the second detected pulse train 520, the start time 525 is at 1 μs, at which time the second pulse train can be emitted. Time between start time 515 and start time 525 can be long enough that any pulses transmitted at the beginning of the first detection interval would have already been detected, and thus not cause confusion with pulses detected in the second detection interval. For example, if there is not extra time between shots, then the circuitry could confuse a retroreflective stop sign at 200 m with a much less reflective object at 50 m (assuming a shot period of about 1 μs). The two detection time intervals for pulse trains 510 and 520 can be the same length and have the same relationship to the respective start time. Time bins 522 and 524 occur at the same relative times of 458 ns and 478 ns as time bins 512 and 514. Thus, when the accumulation step occurs, the corresponding counters can be added. For instance, the counter values at time bin 512 and 522 can be accumulated or added together.

[0049] For the third detected pulse train 530, the start time 535 is at 2 μs, at which time the third pulse train can be emitted. Time bin 532 and 534 also occur at 458 ns and 478 ns relative to start time 535. The counts for corresponding pulses of different pulse trains can have different values even though the emitted pulses have a same power, e.g., due to the stochastic nature of the scattering process of light pulses off of objects.

[0050] Histogram 540 shows an accumulation of the counts from three detected pulse trains 510, 520, 530 at time bins 542 and 544, which also correspond to 458 ns and 478 ns.Histogram 540 can have fewer time bins than were measured during the respective detection intervals, e.g., as a result of dropping time bins in the beginning or the end of the detection interval or time bins having values less than a threshold. In some implementations, about 10-30 time bins can have appreciable values, depending on the pattern for a pulse train,

[0051] As examples, the number of pulse trains emitted during a measurement to create a single histogram can be around 1-40 (e.g., 24), but can also be much higher, e.g., 50, 100, 500, or 1000. Once a measurement is completed, the counts for the histogram can be reset, and another set of pulse trains can be emitted to perform a new measurement. In various embodiments and depending on the number of detection intervals in the respective measurement cycles, measurements can be performed, e.g., every’ 25, 50, 100, or 500 ps. In some embodiments, measurement intervals can overlap, e.g., so that a given histogram corresponds to a particular sliding window of pulse trains. In such an example, memory can be provided for storing multiple histograms, each corresponding to a different time window. Any weights applied to the detected pulses can be the same for each histogram, or such weights could be independently controlled.3, Pixel Operation

[0052] The high photon detection efficiency (PDE), timing resolution, and low dark count rate (DCR) of modern-day SPADs make them suitable candidates for a variety of applications, including ToF applications. A design difficulty in such ToF systems is the requirement for in-pixel storage memory'. These memories can be implemented using d-type flip-flops or other circuits. Much effort has been expended decreasing the size of these circuits to improve scalability. Generally, such designs employ a limited number of bits coupled with off focal plane sub-frame accumulation and / or time to saturation techniques to boost the dynamic range. Direct ToF (DTOF) pixels focus on the efficient usage of limited memory’ resource storing time stamps or partial histogram data. Gating or zooming techniques can be used to capture information over a full measurement range.

[0053] Accordingly, embodiments of the present in vention can provide high density SRAM based pixels. These pixels can be used for multi -bin histograms for DTOF measurements. These pixels can also be used to provide a multi-frame store for intensity imaging. Using an SRAM can provide an order of magnitude increase in storage capacity compared to astandard cell d-type flip-flop in the same technology node. These pixels can include peripheral circuitry to perform precharge-read-increment-write (PRIW) operations. In these and other embodiments of the present invention, an asynchronous SPAD event-driven PRIW operation implemented via an in-pixel timing generator can be utilized. Furthermore, a compact realization of the increment logic can be implemented via an implicit linear 18-bit feedback shift register (LFSR) with XNOR feedback,

[0054] The SRAM word addressing can be operated at pixel row level allowing reprogrammable array operation. When operated at different time-scales, the memory can serve different applications. At nanosecond time-scales, coarse TOF information can be collected and stored in-pixel where memory locations serve as time bins. At microsecond or millisecond time-scales, high speed photon counting enables observing fast occurring phenomena where memory locations serve as a burst or a frame storage before readout is initiated. An example of a pixel using an SRAM is shown in the following figure.

[0055] FIG. 6 illustrates a pixel utilizing an SRAM according to an embodiment of the present invention. In this example, a 21.5μm pixel can integrate an SRAM macro of 16×18 bits. This can enable 16 different storage locations with a counting capacity of 18-bits which can allow harnessing the SPAD’s native dynamic range. This pixel block diagram illustrates SPAD driven memory control and the XNOR LFSR feedback with implicit shift + write operation achieved by wiring the write drivers to the next bitline bus.

[0056] PMOS precharge cells 620 can ensure the internal bit-lines are charged up before addressing. Differential sense amplifiers (SAs) 650 can sample the state of the bit-lines when SRAM 630 is addressed. The sense amplifiers 650 SAs can then feed tri-stated write drivers 670 that can write back into the addressed word. Outputs of sense amplifiers 650 SA<6> and SA<17> can also be provided to XNOR gate 660 that can perform the LFSR code shift. A global 16-bit one-hot code can broadcast across the chip during operation to provide a global address on lines 642 to SRAM 630. The rate of change of the global address on lines 642 depends on the target application and can vary’ between nanoseconds and milliseconds for coarse time resolved (ToF) and multi-frame (imaging) modes respectively.

[0057] The SPAD front end can include a thick oxide PMOS transistor 618 for passive quench and recharge. The cathode of SPAD 610 can be interfaced directly with a thin oxide logic multiplexer 614. The deadtime of SPAD 610 can be controlled by a global VQ bias voltage at the gate of PMOS transistor 618. VQ can be set to provide different durations ofdeadtime as a compromise between dynamic range and the timing margin required for the SPAD driven memory operation. For example, the deadtime can be set to be approximately 5 ns, 7 ns, 10 ns, or other duration. SPAD 610 can reside fully on a top tier of a stacked sensor and can be connected to bottom tier circuitry through a single hybrid-bond site 611.

[0058] Pixel 600 can operate in at least two types of modes. In a first mode, pixel 600 can be used in multi-bin histogramming for time-of-flight measurements. In a second mode, pixel 600 can be used for multi -frame storage intensity imaging. The operation of pixel 600 in each mode can be similar. In each mode, the timing of pixel 600 can be triggered by SPAD 610 as opposed to a clock.

[0059] Specifically, SPAD 610 can detect a photon and thereby driving a voltage at node 612 towards -VHV. This decrease in voltage can activate precharge PMOS cells 620 such that pixel cells of SRAM 630 can be read. Global address lines 642 can be provided to address latch and enable circuits 640 in some or all the pixels 600 in a system. Address latch and enable circuits 640 can provide addresses on lines 644 to SRAM 630, The addresses provided on global address lines 642 can be provided at a high rate, for example every few nanoseconds for ToF binning, or at a slower rate, such as microsecond or millisecond ranges for intensity imaging. In this example address lines 642 and lines 644 can be 16 bits wide to select one of the 16 words 632 in SRAM 630. In these and other embodiments, address line 642 can be encoded to 4 bits and decoded to 16 bit wide addresses on lines 644. In these and other embodiments of the present invention, other numbers of address lines that address different number of words 632 can be implemented.

[0060] Following a SPAD event, SRAM 630 can select a word 632 based on the global address on lines 642. The selected word 632 can be read by sense amplifiers 650, An implicit linear 18-bit feedback shift register (LFSR) with XNOR gate 660 feedback can be used to increment the selected word 632. The selected word 632 can be written back to SRAM 630 by write drivers 670. The timing of the read operation by sense amplifiers 650, the LFSR, and write drivers 670 can be controlled by timing generator 680. Timing generator 680 can be triggered by a SPAD event toggling multiplexer 614 and inverter 616. Following a SPAD event, transistor 618 can be used to recharge SPAD 610.

[0061] More specifically, before a SPAD event occurs, SRAM 630 can be idle and the differential bit-lines 639 can be held in precharge state. Meanwhile, the row driven global address on lines 642. which can be a one-hot code, can be toggling over time. Following aSPAD event, the bit-lines 639 can be released from precharge and the global one-hot code can be sampled into the address latches 640 such that one memory word 632 in SRAM 630 can be addressed. Simultaneously, the buffered SPAD pulse can be passed into timing generator 680. Timing generator 680 can be constructed from delay cells and can have a global delay control voltage VCNTRL. The timing generator can then generate three consecutive pulses to trigger the memory operation, In these and other embodiments of the present invention, the memory operation can occur within the SPAD deadtime

[0062] First, a read pulse can be generated allowing for the SRAM bit-lines 639 to develop a differential voltage which can be sampled by the sense amplifiers onto local capacitors. The sampled state can represent the previous LFSR value stored in SRAM 630. The values in sense amplifiers 650 SA<6> and SA<17> can be used by XNOR gate 660 to generate the next ‘ 1 ’ or ‘0’ bit to be shifted into the LFSR.

[0063] Second, a write pulse can be generated which can activate the differential tristate write drivers 670 to write back the new LFSR code into the same addressed memory word 632. The write operation itself can perform the LFSR code shift implicitly as every wri te driver can be hard wired onto the next SRAM cell bit-lines 639. For example, write driver 670 <0> can wired onto bit-lines 639 <1>, write driver 670 <1> can be wired onto bit-lines 639 <2> and so on. An exception can be write driver 670 <17> which, unlike the rest, can take in the XNOR gate 660 output as its input and can be wired back onto bit-lines 639 <0>. This wiring configuration is shown by dashed lines 671. After the write operation is complete, a reset pulse can be generated to release the address latch 640 such that no memoryword is addressed. Pixel 600 can now be ready for subsequent SPAD 610 events to trigger the precharge-read-increment-write cycle (tire SPAD driven memory operation.)

[0064] Once data capture is complete, data can be read from SRAM 630. Pixel 600 can be configured in readout mode by setting the global Mode signal received by multiplexer 614 high to switch the pixel input to accept external pulses rather than an output from SPAD 610. The Read signal provided to sense amplifiers 650 can be set high in the typical row-by-row rolling shutter fashion. Once a row is selected for readout, the global address on line 642 can be set to select the first memory word. To commence the readout process, an external pulse can then be injected into the pixel triggering an SRAM PRIW. This can result in the previous word value to be sampled into the sense amplifiers 650.[0065| At this point the value of sense amplifier 650 SA<17> can be buffered by the single tri-state read driver 690 onto the column readout line 692 and can be sampled at the edge of the array into a column parallel readout pipeline. Injecting another pulse results in another PRIW operation but due to the shift operation the previous value in sense amplifier 650 SA<16> can appear in sense amplifier 650 SA<17> and can be buffered onto the column output line 692. Repeating this process for a total of 18 pulses (in this example) results in a full readout of the original LFSR code that was stored in the selected memory location.

[0066] Once the word readout is complete, write drivers 670 can write all zeros into the selected memory word to reset it using a row driven reset pulse. Similarly, the global address on line 642 can be updated to select the next memory word 632 and another 18 pulses (in this example) can be injected to complete the readout of the new address. This procedure can be repeated a total of 16 times per row to read out the full memory macro before the next row is selected. The LFSR codes can now be decoded into their equivalent decimal counts.

[0067] When pixel 600 is used in a ToF application, the global address on lines 642 can be updated at a fast rate, where each address is present for only a number of nanoseconds before changing. When an SPAD event occurs, the word 632 identified by the global address on lines 642 can be incremented. Often there is only time for one SPAD event to be counted per emitter cycle, and several emitter cycles can be used to complete the ToF histogram data. In some circumstances, more than one SPAD event can be counted per emitter cycle. This can be spread across the bins. For example, where a bin is 20 nanoseconds and SPAD deadtime is 5 nanoseconds, more than one SPAD event could be counted per bin. When pixel 600 is used in an imaging application, the global address on lines 642 can be updated at a slower rate, where each address is present for a number of microseconds or even milliseconds before changing. This can allow several SPAD events to be counted for a word 632 selected by an address on global address lines 642.

[0068] Pixel 600 can operate in at least these two types of modes. In the first mode, pixel 600 can be used in multi-bin histogramming for time-of-flight measurements and the global address can be updated more frequently. In this first mode, the global address can be changed at least once in the time necessary for 5 SPAD 610 events and corresponding precharge-read-increment-write cycles (the SPAD driven memory operations) to occur. In these and other embodiments of the present invention, in the first mode, the global address can be changed at least once in tire time necessary for 1, 2, 3, 4, or more than 5 SPAD 610events and corresponding precharge-read-increment-write cycles to occur. In this first mode, the global address on lines 642 can be changed in less than 1 ns, approximately 1 ns, less than 5 ns, less than 10 ns, less than 20 ns, less than 50 ns, less than 100 ns, or other duration.

[0069] In a second mode, pixel 600 can be used for multi -frame storage intensity imaging. In this second mode, the global address can be changed no more than once in the time necessary for 50 SPAD events and their corresponding precharge-read-increment-write cycles (the SPAD driven memory operations) to occur. In these and other embodiments of the present invention, in the second mode, the global address on lines 642 can be changed no more than once in the time necessary for 10, 20, 30, 40, 50, 100, or more than 50 SPAD events and their corresponding precharge-read-increment-write cycles to occur. In this second mode, the global address on lines 642 can be changed every 1 microsecond, every 100 microsecond, every’ 1 millisecond, or other duration. The global address on lines 642 can be changed no more than once every microsecond, every 10 microseconds, every 100 microseconds, every millisecond, or other duration. An example showing the timing of pixel 600 is shown in the following figure,

[0070] FIG. 7 illustrates a timing diagram of the global shutter pixel operation. When a SPAD event occurs, the current value in the addressed memory is sampled, incremented by one through an LFSR code shift, then written back into the same memory location. In these and other embodiments of the present invention, this can be done within the SPAD deadtime. In this example, a SPAD event releases the memory precharge signal and samples the global address. The timing generator can split the buffered SPAD pulse into the Read and Write signals required to drive the addressed memory'.

[0071] Timing diagram 700 illustrates the timing for pixel 600. A SPAD event can occur dropping the voltage at a cathode of SPAD 610 (all references shown in FIG. 6) at time 710. SPAD 610 can be recharged as shown by waveform portion 720. The change in voltage at the cathode of SPAD 610 can generate a precharge pulse 730 that can be provided to precharge PMOS cells 620. A global address 740 on lines 642 can be latched by address latch and enable circuit 640 and provided as latched address 742 on lines 644.

[0072] The change in voltage at the cathode of SPAD 610 can further generate a buffered pulse 750 to timing generator 680. Timing generator 680 can generate read signals 760 that can be provided to sense amplifiers 650 and write signals 770 that can be provided to writedrivers 670. After completion, a reset signal 780 can be applied such that write drivers 670 can write all zeros into the selected memory word 632.

[0073] In the above examples, illustrative sizes of SRAM 630, a number of bit lines 339, and illustrative values are shown. In these and other embodiments of the present invention, other sizes of SRAM 630, including the number of words 632, the number of bit lines 639, and others can be varied.

[0074] FIG. 8 shows an intensity image 800 captured by a sensor that includes several pixels 600. Meta-stability glitches manifesting can appear as white speckles.

[0075] FIG, 9 demonstrates the sensor in high speed (burst) imaging mode where a 60 kHz flickering LED was imaged at 16 intervals, each stored in a separate memory location. Each frame 910 can represent a slice of time. Each pixel 920 in each frame 910 corresponds to a word 632 in SRAM 630. Each pixel 600 can store 16 values, one in each word 632 in SRAM 630, where each word 632 is used for a corresponding pixel 920 in each frame 910, A number of pixels 600 can be used to generate each of the frame 910 to complete a series 900 of frames 910. Graphic 930 can indicate the number of SPAD events at each pixel 920 in each frame 910. This number of SPAD events can be determined for each pixel 920 in each frame 910 during a time -where one global address on lines 644 is latched by address latch and enable circuit 640.4. Additional Embodiments

[0076] In the above detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments of the present disclosure. However, it will be understood by those skilled in the art that the present disclosure can be practiced without these specific details. For example, while various embodiments set forth above described can use different numbers of cycles and different power levels, these and other embodiments can use still other numbers of cycles and different power levels. Also, incident photons can be detected for various numbers of time bins that can extend to different time bins. As another example, some of the embodiments discussed above include a specific number of regions or diodes in a SPAD device. It is to be understood that those embodiments are for illustrative purposes only and embodiments are not limited to any particular number of regions or diodes in a SPAD device.

[0077] Additionally, in some instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present disclosure. It is intended that all embodiments disclosed herein can be implemented separately or combined in any way and / or combination. Aspects described with respect to one embodiment can be incorporated in different embodiments although not specifically described relative thereto. That is, all embodiments and / or features of any embodiments can be combined in any way and / or combination.

[0078] The above description of embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above, The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications to thereby enable others skilled in the art to best utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. Thus, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A pixel comprising:a memory comprising a plurality of memory locations;a Single-Photon Avalanche Diode (SPAD); andan address latch circuit having a latch input coupled to the SPAD, an address input coupled to receive a global address, and an address output to select a word in the memory,wherein the pixel is operable in a first mode where tire plurality of memory locations serve as time bins, and in a second mode wherein the plurality of memory locations serve as frame storage.

2. The pixel of claim 1 wherein the memory is a static random-access memory (SRAM).

3. The pixel of claim 2 wherein when the pixel operates in the first mode, the pixel operates at the nanosecond time-scale and when the pixel operates in the second mode, the pixel operates at the microsecond or microsecond time-scale.

4. The pixel of claim 3 further comprising a plurality of sense amplifiers to read a selected word from the SRAM, a linear feedback shift register to increment the selected word, and a plurality of write drivers to write the incremented selected word back to the SRAM.

5. The pixel of claim 4 wherein timing signals for the sense amplifiers and write drivers are provided by a timing generator having an input coupled to the SPAD.

6. The pixel of claim 5 wherein the global address comprises a one-hot code, and wherein the address latch circuit is triggered by an event on the SPAD.

7. The pixel of claim 6 wherein for each of a plurality of write drivers, data is received from a corresponding bit line and data is written back to the next higher bit line.

8. The pixel of claim 7 wherein the linear feedback shift register is implemented for N bits by writing the first N-1 bits back to the next higher bit line along with an exclusive NOR gate.

9. The pixel of claim 8 wherein in the first mode, the global address changes at least once in the time necessary for 5 SPAD events in the second mode the global address changes no more than once at least once in the time necessary' for 50 SPAD events.

10. A method of operating a pixel comprising:providing a global address to an address latch;detecting a Single-Photon Avalanche Diode (SPAD) event;using the detected SPAD event to latch the global address in the address latch; selecting a word in a memory using the latched address;reading the word from the memory;incrementing the word from the memory; andw'riting the word back to memory’.

11. The method of claim 10 wherein providing a global address comprises providing a one-hot code,12. The method of claim 10 wherein selecting a word in a memory comprises selecting a word in a static random-access memory (SRAM).

13. The method of claim 12 further comprising reading the selected w'ord from the SRAM using a plurality of sense amplifiers, incrementing the selected word using a linear feedback shift register, and writing the incremented selected word back to the SRAM using a plurality of write drivers.

14. The method of claim 13 wherein for each of a plurality of write drivers, data is received from a corresponding bit line and data is written back to the next higher bit line.

15. The method of claim 14 wherein the reading, incrementing, and writing acts are triggered by the detected SPAD event.

16. A pixel comprising:a memory comprising a plurality of memory locations;a Single-Photon Avalanche Diode (SPAD); andan address latch circuit having a latch input coupled to the SPAD, an address input coupled to receive an address, and an address output to select a memory location in the plurality of memory locations,wherein the pixel is operable in a first mode and a second mode, and wherein when the pixel operates in the first mode, the pixel operates at the nanosecond time-scale and when the pixel operates in the second mode, the pixel operates at the microsecond or microsecond time-scale.

17. The pixel of claim 16 wherein the memory comprises a static randomaccess memory (SRAM).

18. The pixel of claim 17 wherein the address latch circuit is clocked by an event on the SPAD.

19. The pixel of claim 18 further comprising a plurality of sense amplifiers to read a selected word from the SRAM, a linear feedback shift register to increment the selected word, and a plurality of write drivers to write the incremented selected word back to the SRAM.

20. The pixel of claim 19 wherein for each of a plurality of write drivers, data is received from a corresponding bit line and data is written back to the next higher bit line.

Images