Pixel circuit with single photon detector and image sensor assembly

Abstract

A pixel circuit includes a single photon detector that outputs count pulses in response to photons detected when enabled. A detector enable circuit enables the single photon detector when a detector enable signal is active and disables the single photon detector otherwise. A counter circuit counts the count pulses. A control circuit controls a duty cycle of the detector enable signal depending on a number of pulses counted in the counter circuit.

Description

[0001] 73954

[0002] 1

[0003] PIXEL CIRCUIT WITH SINGLE PHOTON DETECTOR AND IMAGE SENSOR ASSEMBLY

[0004] The present disclosure relates to a pixel circuit that includes a single photon detector outputting count pulses in response to detected photons and a counter circuit that counts the count pulses. The present disclosure further relates to an image sensor assembly that includes a pixel array of pixel circuits having single photon detectors.

[0005] BACKGROUND

[0006] Single photon pixel arrays include a plurality of single photon detectors (SPDs), e.g., single photon avalanche diodes (SPADs) formed in a semiconductor body along pixel rows and pixel columns. Each SPAD includes a photosensitive region. In the operating mode, two opposite electrodes generate an electric field across the photosensitive region, wherein the electric field is approximately as high as the avalanche breakdown electric field. Each photon of sufficient energy entering the photosensitive region generates an electron-hole pair. The electric field separates the generated charge carriers according to their polarity. Since the bias voltage is approximately as high as or higher than the avalanche breakdown voltage, the generated charge carriers generate further charge carriers that induce an avalanche current through the SPAD.

[0007] When the electric field is strong enough to drive the avalanche breakdown into its saturation range, the SPAD operates in Geiger mode. After the avalanche breakdown, a quench mechanism reduces the electric field in the photosensitive region to stop the avalanche breakdown. The quenching resets the SPAD. A comparator circuit transforms the avalanche current spikes into count pulses.

[0008] SUMMARY

[0009] The number of avalanche breakdowns detected during an exposure period corresponds to the number of detected photons and increases with the detected radiation intensity. Typically, each single photon detector pixel circuit includes a counter circuit that counts the count pulses generated within an exposure period. The more counter stages the counter circuit has, the higher the dynamic range of the pixel circuit. The fewer counter stages the counter circuit has, the smaller the area required for the pixel circuit and the higher the spatial resolution can be.

[0010] The present disclosure mitigates such deficiencies of the prior art. In particular, the present disclosure relates to single photon detector pixel circuits whose sensitivity adapts in-situ to the light intensity.

[0011] Accordingly, a pixel circuit includes a single photon detector that outputs count pulses in response to photons detected when enabled. A detector enable circuit enables the single photon detector when a detector enable signal is active and disables the single photon detector otherwise. A counter circuit counts the count pulses. A control circuit controls a duty cycle of the detector enable signal depending on a number of pulses counted in the counter circuit.73954

[0012] 2

[0013] The duty cycle of the detector enable signal is the fraction of the active time of the detector enable signal DEN and defines the fraction of time the single photon detector is active during the exposure period. The lower the duty cycle, the smaller are the time windows in which the single photon detector 110 can detect photons. By adapting the duty cycle of the detector enable signal DEN, the control circuit can adapt in-situ the sensitivity of the pixel circuit to the light intensity.

[0014] BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A more complete understanding of the disclosure and many of the advantages associated therewith will be obtained by reference to the following detailed description in conjunction with the accompanying drawings, in which:

[0016] FIG. 1 is a schematic block diagram of a pixel circuit having a single photon detector that detects photons when a detector enable signal is active, and a control circuit controlling a duty cycle of the detector enable signal depending on the number of detected photons, in accordance with an embodiment.

[0017] FIG. 2 is a simplified circuit diagram illustrating a configuration example of a pixel circuit that includes a SPAD and a detector enable circuit for reducing a voltage across the SPAD when a detector enable signal is inactive, in accordance with an embodiment.

[0018] FIG. 3 is a simplified circuit diagram illustrating a configuration example of a pixel circuit that includes a counter circuit and a control circuit including a line decoder in accordance with an embodiment.

[0019] FIG. 4 is a simplified circuit diagram illustrating a configuration example of a pixel circuit that includes a counter circuit and a control circuit including a 3:8 line decoder in accordance with an embodiment.

[0020] FIG. 5 is a simplified time diagram for digital auxiliary signals having different duty cycles usable as sources for the detector enable signal, in accordance with an embodiment.

[0021] FIG. 6 is a simplified circuit diagram illustrating a configuration example of a pixel circuit that includes a pixel memory in accordance with an embodiment.

[0022] FIG. 7 is a simplified block diagram illustrating components of a pixel circuit and a signal source that provides digital auxiliary signals with different duty cycles for an image sensor assembly, in accordance with an embodiment.

[0023] FIG. 8 is a simplified time diagram for signals controlling a transfer of a count value of the counter circuit from the pixel circuit to an associated pixel memory, in accordance with an embodiment.

[0024] FIG. 9 is a simplified block diagram illustrating a configuration example of an image sensor assembly that includes single photon detector pixel circuits, in accordance with an embodiment.3

[0025] FIG. 10 is a simplified time diagram for parallel readout of a plurality of memory array rows of the image sensor assembly of FIG. 9, in accordance with an embodiment.

[0026] FIG. 11 is a simplified time diagram for digital auxiliary signals having different duty cycles in relation to a memory write time in accordance with an embodiment.

[0027] FIG. 12 is a simplified circuit diagram illustrating a configuration example of a pixel circuit that controls a horizontal interrupt in accordance with an embodiment.

[0028] FIG. 13 is a simplified block diagram illustrating a configuration example of an image sensor assembly that includes single photon detector pixel circuits as illustrated in FIG. 12, in accordance with an embodiment.

[0029] FIG. 14 is a simplified time diagram for readout of the pixel circuits of FIG. 12 in accordance with an embodiment concerning a rolling shutter operation.

[0030] FIG. 15 is a simplified time diagram for readout of the pixel circuits of FIG. 12 in accordance with an embodiment concerning a global shutter operation.

[0031] FIG. 16 is a schematic diagram illustrating an embodiment in which an image sensor has a two-layer structure in a stacked CIS configuration.

[0032] FIG. 17 is a block diagram depicting an example of a schematic configuration of a vehicle control system.

[0033] FIG. 18 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section of the vehicle control system of FIG. 17.

[0034] DETAILED DESCRIPTION

[0035] Embodiments for implementing techniques of the present disclosure will be described below in detail using the drawings. The techniques of the present disclosure are not limited to the described embodiments, and various numerical values and the like in the embodiments are illustrative only. The same elements and elements with the same functions are denoted by the same reference signs. Duplicate descriptions are omitted.

[0036] The terms “electrically connected”, “signal-connected”, and “operatively connected” may include a direct connection or a connection through other electronic elements provided and suitable for permanent and / or temporary signal transmission and / or transmission of energy. Electronic elements can be electrically connected, signal-connected and operatively connected via resistors, capacitors, electronic switches such as field effect transistors, or transistor circuits such as transmission gates. At least one electrical signal in a second electrical circuit that is in signal-connection or operational connection with a first electrical circuit responds in a predictable, intended manner to a change of an electrical signal in the first electrical circuit.4

[0037] Directly electrically connected electronic elements are connected through a permanent low-resistive wiring, an ohmic contact and / or a unipolar semiconductor junction.

[0038] The load path of a transistor is the controlled current path through a transistor. A voltage applied to the gate of a field effect transistor (FET) controls the current flow through the load path (controlled path) between source and drain of the transistor by field effect. When transistors are electrically connected in series, the load paths of the transistors are electrically connected in series. When transistors are electrically connected in parallel, the load paths of the transistors are electrically connected in parallel.

[0039] A digital signal alternates between at least one active level and at least one inactive level. A digital signal having an active level is active. A digital signal having an inactive level is inactive. For each signal separately, the active level can be a digital high level and the inactive level a digital low level, or the active level can be the digital low level and the inactive level the digital high level. Digital signals include binary signals alternating between one active level and one inactive level.

[0040] FIG. 1 shows a pixel circuit 100 that includes a single photon detector 110 configured to output count pulses in response to photons detected when enabled. A detector enable circuit 120 is configured to enable the single photon detector 110 when a detector enable signal DEN is active and to disable the single photon detector 110 otherwise. The pixel circuit 100 further includes a counter circuit 130 configured to count the count pulses, and a control circuit 140 configured to control a duty cycle of the detector enable signal DEN depending on a number of pulses counted in the counter circuit 130.

[0041] When the single photon detector 110 is enabled or activated, an incoming photon triggers an avalanche breakdown, and an avalanche current begins to flow. The avalanche breakdown causes a detector output signal to change from inactive to active, i.e., from an inactive signal level to an active signal level. After detection of the avalanche breakdown, the avalanche current is interrupted, the avalanche breakdown is terminated, the single photon detector 110 returns to the initial reverse-biased blocking state, and the detector output signal returns to the inactive signal level. For each detected photon, the single photon detector 110 outputs a count pulse with a pulse width corresponding to the time the detector output signal is active.

[0042] The detector enable signal DEN controls the detector enable circuit 120, wherein an active detector enable signal DEN enables the single photon detector 110 for active periods and an inactive detector enable signal DEN disables the single photon detector 110 outside the active periods. When enabled, a) the single-photon detector 110 is reverse biased with a voltage close to, e.g., higher than the breakdown voltage so that an incoming photon triggers an avalanche breakdown, b) the avalanche current through the single photon detector 110 causes a count pulse, and c) the counter circuit 130 receives and counts the count pulses output by the single photon detector 110. When disabled, at least one of the requirements a, b and c is not fulfilled.

[0043] The counter circuit 130 may be a resettable synchronous or asynchronous digital counter having a plurality of digital counter stages. The counter stage for the least significant bit may receive the count pulses at a clock input of the counter circuit 130. Each following counter stage receives the output signal of the directly5

[0044] preceding counter stage. The number of counter stages may be at least six, at least eight or at least ten. The number of counter stages may be at most sixteen, at most twelve, at most ten, or at most eight. An active reset signal may set the outputs of all counter stages to inactive, e.g., logic “0”. When the reset signal changes to inactive, the counter circuit 130 begins to count the count pulses received.

[0045] The total output status of all counter stages represents a digital count value that indicates the number of pulses received by the counter circuit 130 after resetting. The number of pulses is equal to the number of detected photons. The number of detected photons is referred to below as the photon count and provides the pixel data. The output signal of each counter stage represents a bit of the digital count value. The counter circuit 130 may include a serial data output port for successively outputting the digital count value via a single port to an associated pixel memory, or a parallel data output port for simultaneously outputting all bits of the digital count value to the associated pixel memory.

[0046] The control circuit 140 receives the output signal of at least one of the counter stages directly from the counter circuit 130 or receives at least one signal derived from one or more of the output signals of the counter stages of the counter circuit 130. For example, the control circuit 140 directly receives the output signal of at least one of the counter stages for the three most significant bits of the count value in the counter circuit 140. This allows the control circuit 140 to control the duty cycle of the detector enable signal DEN as a function of the current digital count value of the counter circuit 130.

[0047] The duty cycle is the fraction of the active time of the detector enable signal DEN and defines the fraction of the time the single photon detector 110 is active during the exposure period. The lower the duty cycle, the smaller are the time windows in which the single photon detector 110 is sensitive and counts the count pulses.

[0048] The control circuit 140 can operate the detector enable circuit 120 as an additional, pixel-specific shutter that locally reduces in-situ the array-wide exposure period defined by a global shutter or an array-wide rolling shutter. In principle, different photon counts can be used as thresholds, wherein for each individual threshold, at the pixel sensitivity can be increased by increasing the duty cycle of the detector enable signal DEN or decreased by decreasing the duty cycle of the detector enable signal DEN, depending on the specifics of an application. A downstream processing unit can use the count values and information about the thresholds to linearize the count values output by the pixel circuit 100.

[0049] For example, the control circuit 140 may be configured to control a step-wise decrease of the duty cycle of the detector enable signal DEN with increasing number of counts.

[0050] In particular, the control circuit 140 may control a monotonic step-by-step decrease of the duty cycle of the detector enable signal DEN with each threshold for the photon count reached within the same exposure period.

[0051] In this way, the sensitivity can be selectively reduced in steps selectively for those pixel circuits 100 that have already reached comparatively high photon counts and for which a counter overflow within the current6

[0052] exposure period is becoming increasingly probable. Since an undefined overflow of the counter circuit 130 means a loss of information, the pixel-internal shutter reduces the probability of an overflow of the counter circuit 130 during array-wide exposure. Instead, a piece-wise linear sensitivity response can be achieved with ever decreasing intensity resolution for increasing intensity values.

[0053] In the pixel circuit 100 illustrated in FIG. 2, the single photon detector 110 includes a single-photon avalanche diode 111 and a quenching unit 115 that includes a quench path electrically connected in series with the single-photon avalanche diode 111.

[0054] The quenching unit 115 may include an active quench element 116, e.g., a field effect transistor with a load path electrically connected in series with the single-photon avalanche diode 111. The load path of the field effect transistor forms the quench path. The quench element 116 and the single-photon avalanche diode 111 may be electrically connected in this order between a high supply voltage VA and a reference potential VSS. In the illustrated example, the active quench element 116 is an n-channel FET.

[0055] The quenching unit 115 may cause a current loop for the avalanche current of the single-photon avalanche diode 111 to be interrupted or a resistance in the current loop for the avalanche current to be significantly increased when the avalanche current rises. As a result of the interrupted avalanche current or the increased voltage drop across the quench path, the voltage across the single-photon avalanche diode 111 falls below a threshold voltage below which incoming photons do not cause an avalanche breakdown. The quenching unit 115 may further include a comparator circuit that outputs a count pulse in response to each avalanche event.

[0056] FIG. 2 further shows a detector enable circuit 120 that includes an electronic switch 125 configured to reduce a voltage across the single-photon avalanche diode 111 when the detector enable signal DEN is inactive.

[0057] When the detector enable signal DEN is active, the electronic switch 125 ensures that a voltage across the single-photon avalanche diode 111 is high enough so that incoming photons can trigger an avalanche breakdown. When the detector enable signal DEN is inactive, the electronic switch 125 ensures that a voltage across the single-photon avalanche diode 111 is low enough to prevent incoming photons from triggering an avalanche breakdown.

[0058] For example, the single-photon avalanche diode 111 may be electrically connected between the high supply potential VA and the reference potential VSS while the single photon detector 110 is enabled, and may be electrically connected between the high supply potential VA and an intermediate potential VDD or between an intermediate potential VDD and the reference potential VSS when disabled, wherein the intermediate potential VDD is selected in such a way that received photons do not trigger an avalanche breakdown in the single-photon avalanche diode 111 at 20 degree Celsius. By avoiding unnecessary avalanche breakdowns, i.e., avalanche breakdowns that are not counted or that may saturate the counter circuit 130, electrical power can be saved.7

[0059] If the single photon detector 110 includes a quenching unit 115 having a quench path electrically connected in series with the single-photon avalanche diode 111, the detector enable circuit 120 may directly or indirectly control the quenching unit 115 to sufficiently separate the single-photon avalanche diode 111 from the high supply potential VA or the reference potential VSS to prevent a short-circuit between the intermediate potential VDD and the high supply potential VA or between the intermediate potential VDD and the reference potential VSS.

[0060] In the illustrated example, the electronic switch 125 includes a p-channel FET with a load path electrically connected between a positive supply potential VDD and the anode of the single-photon avalanche diode 111. The cathode of the single-photon avalanche diode 111 is connected to the high supply potential VA. The active level of the detector enable signal DEN is the digital high level and the inactive level is the digital low level. When the detector enable signal DEN has the digital high level, the electronic switch 125 is “off’. As long as no avalanche breakdown occurs, the n-channel FET forming the active quench element 116 is “on” and connects the anode of the single-photon avalanche diode 111 to the reference potential VSS. The voltage difference VA-VSS is sufficiently high so that incoming photons can trigger avalanche breakdowns. When the detector enable signal DEN has the digital low level, the anode of the single-photon avalanche diode 111 is connected via the p-channel FET of the electronic switch 125 to the intermediate potential VDD. The high potential at the input of the comparator circuit 117 effects the comparator circuit 117 to switch off the n-channel FET forming the active quench element 116. The resulting voltage difference AV = VA - VDD across the single-photon avalanche diode 111 is sufficiently low so that incoming photons do not trigger avalanche breakdowns.

[0061] The control circuit 140 of FIG. 1 may include an astable multivibrator with one or more switches to select different capacitances for facilitating different duty cycles for an in-pixel generated detector enable signal DEN.

[0062] In FIG. 3, the counter circuit 130 receives the count pulses at a clock input elk. An active reset signal rst resets all m counter stages, for example, to logic “0”. After the reset signal rst changes to inactive, the counter circuit 130 begins to count the count pulses. The counter stages for the n most significant bits output the n most significant bits at outputs q(m-n) to q(m-l).

[0063] In the illustrated example, the control circuit 140 is configured to connect an input of the detector enable circuit 120 with one of at least two global routing lines 241 in a first phase and with another one of the at least two global routing lines 241 in a second phase.

[0064] On the at least two global routing lines 241, digital auxiliary signals with different duty cycles can be transmitted from a signal source to all pixel circuits 100 of a pixel array. The control circuit 140 connects the input of the detector enable circuit 120 to one of the global routing lines 241, depending on the number of pulses already counted in the counter circuit 130. For low count values, the control circuit 140 may select a global routing line 241 on which a digital auxiliary signal with high duty cycle is transmitted. For higher count values, the control circuit 140 may select global routing lines 241 on which digital auxiliary signals with lower duty cycles are transmitted.8

[0065] More particularly, the illustrated control circuit 140 includes 2An routing switches 141 configured to be controlled by output signals of counter stages for the n most significant bits of the counter circuit 130, wherein each routing switch 141 is configured to connect an input of the detector enable circuit 120 with one of 2An global routing lines 241 , with n being a positive integer number.

[0066] By supplying digital auxiliary signals dO, ... d2An-l having different duty cycles to the 2An global routing lines 241, it is possible to obtain a piece-wise linear resolution for the intensity of received light. For example, the resolution may decrease towards higher intensities in 2An steps, wherein the step height may be constant or may monotonically increase with increasing intensity.

[0067] The illustrated control circuit 140 further includes a n-to-2An line decoder 149 configured to control the 2An routing switches 141 in response to the output signals of the counter stages for the n most significant bits.

[0068] In the illustrated example, the counter circuit 130 includes m counter stages 0 to m-1. The outputs q(m-n) to q(m-l) output the output signals of the counter stages for the n most significant bits of the photon count. The line decoder 149 receives the n output signals of the counter circuit 130 at n decoder inputs aO to a(n-1) and outputs 2An switch control signals for controlling the 2An routing switches 141 at 2An decoder outputs qO to q(2An-l).

[0069] FIG. 4 shows an example for a pixel circuit 100 including a counter circuit 130 with ten counter stages and a 3:8 line decoder 149 that receives the three most significant bits of the full photon count including leading zeroes to control eight routing switches 141. For example, as long as the three most significant bits are equal 0 (digital value “000”), a first output qO of the line decoder 149 is active and controls the first routing switch 141 to connect the input of the detector enable circuit 120 with the global routing line 241 that transmits the digital auxiliary signal dO with the highest duty cycle. When the three most significant bits represent a digital value “001”, a second output ql of the line decoder 149 is active and controls the second routing switch 141 to connect the input of the detector enable circuit 120 with that global routing line 241 that transmits the digital auxiliary signal dl with the second highest duty cycle. When the three most significant bits represent a digital value “111”, an eighth output q7 of the line decoder 149 is active and controls the eighth routing switch 141 to connect the input of the detector enable circuit 120 with that global routing line 241 that transmits the digital auxiliary signal d7 with the lowest duty cycle.

[0070] FIG. 5 refers to an example with the global routing lines 241 being configured to transmit digital auxiliary signals dO, dl, ... , wherein at least one of the digital auxiliary signals dO, dl, ... , is a periodic signal with a period length equal to or shorter than a shortest exposure period.

[0071] For example, the global routing lines 241 are temporarily connectable or permanently connected to outputs of a signal source providing the digital auxiliary signals dO, dl, ... . The signal source may be provided inside or outside the pixel circuit 100.9

[0072] Each of the 2An digital auxiliary signals dO to d(2An-l) may contain a sequence of two or more pulses with the same or different pulse length, wherein the digital auxiliary signal is active for the pulse length. The digital auxiliary signals dO to d(2An-l) may include sequences of pulses having different pulse lengths and the same repetition rate frequency and / or sequences of pulses having the same pulse length and different repetition rates frequencies.

[0073] More specifically, FIG. 5 relates to a configuration example with at least some of the digital auxiliary signals dO to d(2An-l) containing periodic pulses of equal pulse length, wherein for different digital auxiliary signals dO to d(2An-l) the pulses have different repetition rates (frequencies).

[0074] For example, all but one of the digital auxiliary signals may contain periodic pulses of equal pulse width, wherein the digital auxiliary signal dO without periodic pulses may be continuously active throughout the entire exposure period to facilitate continuous photon counting for a standard mode in which the single photon detector is permanently enabled in the exposure period.

[0075] Due to the periodic nature of the digital auxiliary signals dO, dl, ..., the times in which the single photon detector 110 can detect photons are evenly distributed over the remainder of the exposure period. The probability that certain events in the scene are missed as a result of too long inactive periods of the single photon detector 110 is reduced.

[0076] In the illustrated example, the duty cycle of the first digital auxiliary signal dO is 100% for the exposure period. The duty cycle of the second digital auxiliary signal dl is about 1:3, and the duty cycle of the third digital auxiliary signal d2 is about 1 : 12. In another example, the duty cycle may decrease by the same factor from one digital auxiliary signal dp to the next digital auxiliar signal d(p+l), e.g., 1:1, 1:5, 1 :25, 1 : 125, .... or 1:1, 1:10, 1:100, ..., or with increasing fraction such as 1:1, 1:2, 1:8, 1:64, ... .

[0077] FIG. 6 shows a pixel memory 170 configured to receive and store the number of pulses counted in the counter circuit 130.

[0078] Each pixel circuit 100 includes a pixel counter unit 150 that includes at least the counter circuit 130 and the control circuit 140, and that may further include the quenching unit 115 and the detector enable circuit 120. The pixel counter unit 150 is signal -connected to the associated pixel memory 170 via one or more point-to-point connections. For example, the pixel counter unit 150 and the pixel memory 170 may be formed on different substrates that may be stacked on top of each other. Provided that the pixel memory 170 has approximately the same footprint as the pixel counter unit 150, placing the pixel counter unit 150 on top of the associated pixel memory 170 facilitates point-to-point connections through bonded metal pads. The point-to-point connections facilitate a simultaneous transfer of the photon counts from the counter circuits 130 of all pixel circuits 100 to the pixel memories 170 (“array write”). For a serial data output from the counter circuit 130, a single data line may provide a single point-to-point connection between the counter circuit 130 and the pixel memory 170. For a parallel data output from the counter circuit 130, a number of data lines corresponding to the number of counter stages in the counter circuit 130 may provide a10

[0079] corresponding number of point-to-point connections between the counter circuit 130 and the pixel memory 170.

[0080] Alternatively, a plurality of pixel circuits 100 in a same pixel column may share a common data line for serial data transfer or may share a common set of data lines for parallel data transfer from the counter circuits 130 to the pixel memories 170. The pixel counter units 150 and the pixel memories 170 may be formed on the same substrate or on different substrates. On the shared data lines, the photon counts are transmitted in a time-multiplexed manner row-by-row.

[0081] FIG. 7 refers to a configuration example of an image sensor assembly 70. The image sensor assembly 70 includes a pixel circuit 100 that includes a control circuit 140 configured to connect an input of the detector enable circuit 120 with one of at least two global routing lines 241 in a first phase and with another one of the at least two global routing lines 241 in a second phase as described with reference to FIG. 3 and FIG. 4. The images sensor assembly 70 further includes a signal source 31 that is configured to generate digital auxiliary signals dO, dl, ... with different duty cycles and pass each of the digital auxiliary signals dO, d 1 , ... to a different one of the global routing lines 241.

[0082] At least some of the digital auxiliary signals dO, dl, ... may contain periodic pulses of equal pulse length, wherein for different digital auxiliary signals dO, dl, ... the pulses have different repetition rates as described with reference to FIG. 5. For example, all but one of the digital auxiliary signals dO, dl, ... may contain periodic pulses of equal pulse width, wherein the digital auxiliary signal without periodic pulses may be continuously active throughout the entire exposure period to facilitate continuous photon counting for a high-sensitivity mode in which the single photon detector is permanently enabled.

[0083] The signal source 31 may be part of or integrated in a row controller 30 that supplies further control signals. In the illustrated example, the row controller 30 further outputs a reset signal rst and a shift signal shft for controlling the counter circuits 130.

[0084] The signal source 31 may be controllable. For example, the duty cycles of one, some or all digital auxiliary signals dO, dl, ... may change in response to a change of a user setting, a change of an application condition, in response to a change of an internal state of the image sensor assembly 70, and / or as a function of intensity information and / or image information obtained from the pixel circuits 100 during operation. Alternatively, the duty cycles of all digital auxiliary signals dO, dl are fixed and independent from image information, user settings, and others.

[0085] As illustrated in FIG. 8, the shift signal shft may be a signal including a sequence of n pulses in a write-and-reset period with a memory write time twr. The n pulses are applied to a shift input of the counter circuits 130 and effect a sequential read out of the output signals of the n counter stages from the counter circuit 130 through a serial data output port, and the sequential write of the bits of the photon count into the associated pixel memory 170. For eight bit data, it takes eight clock cycles (8 pulses) to transfer the data. After transmission of the photon count to the pixel memory 170, the row controller 30 may output an active reset signal rst that clears the counter stages. For example, the active reset signal rst sets all count values in73954

[0086] 11

[0087] the counter circuits 130 to “logic 0”. When the reset signal rst changes to inactive, the memory write time twr ends and an exposure period with the exposure time texp begins.

[0088] FIG. 9 is related to an image sensor assembly 70 that includes a pixel array 10. The pixel array 10 includes a plurality of pixel rows 11. Each pixel row 11 includes a plurality of pixel counter units 150 of the pixel circuits 100, wherein each pixel counter unit 150 includes at least the counter circuit 130 of the pixel circuit 170. The image sensor assembly 70 further includes a memory array 20 that includes pixel memories 170 configured to receive and store the number of pulses counted in the counter circuits 130.

[0089] All pixel rows 11 may include the same number of pixel counter units 150. The pixel memories 170 are assigned to a plurality of memory rows 21. All memory rows 21 may include the same number of pixel memories 170. The number of pixel counter units 150 per pixel row 11 and the number of pixel memories 170 per memory row 21 may be equal N. The number of pixel rows 11 and the number of memory rows 21 may be equal M so that each pixel memory 170 is associated to one pixel counter unit 150, wherein the pixel counter unit 150 and the pixel memory 170 form a pixel circuit 100. Alternatively, the number of memory rows 21 may be smaller than the number of pixel rows 11 so that each pixel memory 170 is associated to a group of pixel counter units 150, wherein one pixel memory 170 is shared by a plurality of a pixel counter units 150.

[0090] The illustrated memory array 20 includes a pixel memory 170 for each of the pixel circuits 100.

[0091] The number of pixel rows 11 and the number of memory rows 21 is equal M. Each pixel memory 170 is associated to one pixel counter unit 150. One pixel counter unit 150 and one pixel memory 170 form a pixel circuit 100.

[0092] The photon counts (pixel data) of all pixel counter units 150 may be written in parallel from all pixel counter units 150 of the pixel array 10 into the pixel memories 170 of the memory array 20. A memory write time twr is very short in relation to a typical exposure period.

[0093] A readout of the pixel data from the memory array 20 to a digital core 50 may include a successive readout of all M memory rows 21 to a M: 1 multiplexer 40, wherein the readout of each memory row 21 includes a successive readout of all N pixel memories 170 in the memory row 21.

[0094] The unit being multiplexed can be one memory row 21 or more than one memory row 21. For example, the multiplexer 40 may route four data paths in parallel to the digital core to process four memory rows 21 in parallel. For each address of the multiplexer 40, four different memory rows 21 can be routed to four data output lines of the multiplexer 40. For example, if the multiplexer address is “0” then the multiplexer 40 may connect memory rows 0 to 3 to the four data output lines, if the multiplexer address is “1” then the multiplexer 40 may connect memory rows 4 to 7 to the four data output lines, and so on.

[0095] The digital core 50 receives one or more serial streams of pixel data in a way that is compatible with internal data pipelines of the digital core 50 and the way the pixel data is processed in the digital core 50. Depending73954

[0096] 12

[0097] on the number of pixels, a comparatively high clock rate may be required for a purely serial readout of M*N pixel data with a data length of eight, ten or twelve bits. The digital core 50 may further process the pixel data and transmit the processed pixel data in a standardized form to an I / O Interface circuit 60. The I / O interface circuit 60 may include a MIPI (mobile industry processor interface), by way of example.

[0098] The illustrated row controller 30 can be configured to control a transfer of the number of counts counted in the pixel circuits 100 of a selected pixel row 11 to the pixel memories 170 associated with the pixel circuits 100 of the selected pixel row 11 in a memory write period following the exposure period for the pixel row 11.

[0099] The row controller 30 may control pixel exposure and the transfer of the photon count from the pixel counter unit 150 to the pixel memory 170 through control signals. A first one of the control signals is the reset signal rst for resetting the counter circuit 130 as described with reference to FIG. 3. Another control signal may be the shift signal for clocking the counter circuit output signal out at a serial data output port as described with reference to FIG. 8. The term “memory write time” refers to the time needed for writing the pixel data into the memory array 20 and resetting the counter circuits 130.

[0100] When the image sensor assembly 70 is in a global shutter configuration, the exposure period is the same for all pixel rows 11. All pixel circuits 100 may receive the same reset signal, the counter circuits 130 of all pixel circuits 100 are reset at the same time, and the count values of the counter circuits 130 of all pixel circuits 100 are transferred to the associated pixel memories at the same time. For example, the pixel data is serially shifted into the pixel memory 170 associated to the pixel circuit 100. The pixel data for all pixel circuits 100 are shifted in parallel. Each pixel circuit 100 may be connected to the associated pixel memory 170 via an own set of pixel-internal data lines.

[0101] When the image sensor assembly is in a rolling shutter configuration, the exposure periods for the pixel rows 11 can be offset in time to each other. The pixel rows 11 may receive different reset signals m, the counter circuits 130 of pixel circuits 100 of different pixel rows are reset at different times, and the count values of the counter circuits 130 of all pixel circuits 100 can be transferred to the associated pixel memories at different times. Pixel circuits 100 of the same pixel column and pixel memories of the same memory array column may share a common data line or a set of common data lines. If a reset pointer identifying the pixel row 11 to be reset is a number T of pixel rows 11 ahead of the pointer identifying the pixel row 11 for data transfer to the corresponding memory row 21, then the exposure time corresponds to the readout time of T pixel rows 11.

[0102] FIG. 10 illustrates a readout of the digital data from the memory array 20 that includes a successive readout of M / P groups of P memory rows 21 from memory row Mem_x to memory row Mem_(x+P-1). Each readout of each memory row 21 includes a successive readout rdO, rdl, ... of all N pixel memories 170 in the same memory row 21. All P memory rows 21 of the same group of memory rows 21 are read out in parallel from memory column 0 to memory column N-l. The row read time trow is N times the column read time tcol. The read clock can be P times slower compared to the case with a purely serial readout. An intermediate buffer may serialize the pixel data that a M / P multiplexer 40 outputs in parallel on P73954

[0103] 13

[0104] multiplexer output lines. Alternatively, the digital core 50 may be configured to receive and process P parallel pixel data streams.

[0105] With tcol being the time to read out one pixel memory 170 of a memory row 21, the total row read time trow for reading out a complete memory row 21 is given by trow = N * tcol, with N being the number of memory columns. The highest possible frame rate is given by l / (trow*M+twr), with M being the number of memory rows and twr the memory array write time of the memory array. The parallel readout of groups of memory rows 21 may relax time constraints of the memory array readout.

[0106] FIG. 11 refers to an example, with the signal source 31 of FIG. 7 being configured to generate a first digital auxiliary signal dO being active for the exposure period and inactive for the memory write period.

[0107] The first digital auxiliary signal dO disables the single photon counter for the memory write period. The transfer of the count value from the pixel circuit 100 to the pixel memory is not affected by photons detected during the memory write period. Irrelevant avalanche breakdowns are avoided. All other digital auxiliary signals dl, d2, ... may be inactive as well.

[0108] The illustrated example refers to a global shutter operation. The exposure period starts and ends simultaneously for all pixel circuits 100. Within the exposure period texp, all M memory rows 21 are read out one after the other, each with a row read time trow.

[0109] In FIG. 12, the pixel circuits 100 of each pixel row 11 are configured to set a horizontal interrupt signal to an active level, when the number of counts counted in the counter circuit 130 exceeds a predefined threshold value.

[0110] For example, the output signal of at least one of the counter stages for the three most significant bits or an overflow signal can directly or indirectly control the horizontal interrupt signal hi. The output signals of all pixel circuits 100 in the same pixel row 11 may be combined, e.g., through a wired OR, so that one single active output signal is sufficient to bring the horizontal interrupt signal hi to an active level, and additional active output signals from other pixel circuits 100 have no further effect. The active level of the horizontal interrupt signal hi can then be the low level.

[0111] In the illustrated embodiment, an active output signal sti of the counter circuit 130 turns on an nFET 161. When any of the nFETs 161 of a pixel row is switched on, the horizontal interrupt signal hi for the pixel row goes low. The counter circuit 130 may reset the output signal sti to the inactive level in response to an active reset signal rst for the pixel row received at an reset input rst.

[0112] A row select signal sel turns on data bus switches 162 for the pixel row for a memory write period. Each data bus switch 162 connects the data output port do of one of the counter circuits 130 of the pixel row to a data bus shared by a plurality of pixel circuits 100 of the same pixel column.73954

[0113] 14

[0114] The counter stages for bits qO to q(m-n-l) on the one hand and the counter stages for bits q(m-n) to q(m-l) on the other hand, i.e., the bits not used for selecting the digital auxiliary signals dO to d(2n-l) and the bits used for selecting the digital auxiliary signals dO to d(2n-l) may be reset simultaneously or separately. For example, the active reset signal rst may be used to reset the counter stages for all bits qO to q(m-l) simultaneously. Alternatively, the active reset signal rst may be used to reset the counter stages for bits q(m-l) to q(m-n), and a rising edge of a pulse of the row select signal sel received at a second reset input rstlb resets the counter stages for the bits q(0) to q(m-n-l) when the pixel data is output through the data output port do. The separate reset allows the previous duty cycle to be used after the readout of a subframe to avoid that the total SPAD energy consumption is higher for a subframe readout scheme compared to a one frame readout scheme.

[0115] The data output port do may be a serial data port, wherein the data bus includes one data line and the pixel circuit 100 includes one data bus switch 162. Alternatively, the data output port do may be a parallel data port, wherein the data bus includes as many data lines as the counter circuit 130 includes counter stages and wherein the pixel circuit 100 includes one data bus switch 162 per data line. For a latched parallel data output port do, a transition of the row select signal sel from inactive to active may latch the current count value into the latch of the parallel data port.

[0116] FIG. 13 shows an image sensor assembly 70 that includes a memory control unit 51 configured to control a readout of a pixel row 11 only when the horizontal interrupt signal for the pixel row 11 is active.

[0117] The memory control unit 51 may be integrated in the digital core 50 and may be in operational connection with the row controller 30 to control the reset signals rsti. Depending on the application and / or operation mode, electrical power can be saved by skipping the readout of pixel rows 11 that have not detected significant light.

[0118] Further, the pixel array 10 includes a number Q of pixel groups 15, each pixel group 15 includes a number P of pixel rows 11, each pixel row 11 includes a number N of pixel counter units 150, and wherein the memory array 20 includes Q memory rows 21 of the pixel memories 170.

[0119] All pixel counter units 150 of the same pixel row 11 are simultaneously selected for readout of the pixel data, i.e., for transfer of the current counter value to one of the Q memory rows 21. Different pixel rows 11 can be selected individually for readout. Each pixel counter unit 150 is connectable to a data line do<k:0> for serial readout, or a set of data lines for parallel readout. The data line for serial readout or the set of data lines for parallel readout is shared by the P pixel counter units 150 of the same pixel column 12 in the same pixel group 15 so that via the same data line(s), the pixel data of P pixel counter units 150 of the same pixel group 15 can be successively written to and read out from the pixel memories 170 of the same memory row 21. For each pixel column, Q separated data lines for serial readout (or sets of data lines for parallel readout) facilitate simultaneous readout from the Q pixel groups 15 to the Q memory rows 21.73954

[0120] 15

[0121] The row controller 30 and / or the digital core 50 control the pixel array 10 and the memory rows 21 such that all pixel memories 170 in a memory row 21 are written in parallel and are read out successively before the next pixel row 11 is written to the memory row 21.

[0122] The image sensor assembly 70 further includes a linearization unit 52 configured to obtain compensated count values from the count values received from the memory array 20, wherein for the compensated count values each count step indicates a same amount of change of received light intensity.

[0123] In the illustrated example, the linearization unit 52 is integral part of the digital core 50. In the compensated count values, the effect of the piece-wise linearization by the pixel-internal shuttering is compensated across the whole range. The compensated count values have more digits than the stored count values. With 8-digit count values, the compensated count values can have 12 digits, for example.

[0124] The image sensor assembly 70 further includes a first frame buffer 53 configured to accumulate pixel data, a second frame buffer 54 configured to accumulate pixel data, and a frame controller 55. The frame controller 55 controls the first frame buffer 53 to accumulate first subframes, wherein each first subframe contains pixel data from the pixel counter units 150 of the pixel array 10 obtained during a first exposure period. The frame controller 55 controls the second frame buffer 54 to accumulate second subframes, wherein each second subframe contains pixel data from the pixel counter units 150 of the pixel array 10 obtained during a second exposure period.

[0125] In order to obtain an image with a high dynamic range (HDR), several subframes of the same scene can be obtained directly one after the other. The subframes may be streamed out directly via an I / O interface circuit 60. The frame buffers 53, 54, on the other hand, allow the HDR images to be generated locally by accumulating the pixel data in the image sensor assembly 70. By transmitting only the complete HDR image via the I / O interface circuit 60 instead of a large number of subframes, data bandwidth can be saved. Each subframe can be read out in a rolling shutter read fashion for the pixel rows 0 to P-1 within each pixel group 25. The frame controller 55 controls a parallel read out of the pixel data from the pixel groups 0 to Q-l and the accumulation of the pixel data in one of the frame buffers 53, 54.

[0126] In particular, for HDR images with a dynamic range of 140dB and more, the frame buffers 53, 54 relax the requirements for the number of bits to be stored in the pixel circuits.

[0127] The final HDR image is only available when the last subframe has been accumulated in a frame buffer. When in case of a single frame buffer the frame buffer is read out for transmitting the final HDR image through the I / O interface circuit 60, the maximum framerate fps is equal to l / (T_write + T_read), where T_write is the total exposure / read / accumulation time for all subframes and T_read is the time it takes to read out the entire frame buffer through the I / O interface circuit 60. This puts a limit on the maximum framerate and the maximum exposure time, which would be S+l sub-frames (to avoid overflow).

[0128] With the two frame buffers 53, 54, it is possible to either write / accumulate into the first frame buffer 53 and simultaneously read from the second frame buffer 54, or write / accumulate into the second frame buffer73954

[0129] 16

[0130] 54 and simultaneously read from the first frame buffer 53. Then there is no limit on the framerate and the exposure time can be the entire frame time. In a ping-pong read / write sequence, writing / accumulating into the first frame buffer 53 and simultaneously reading from the second frame buffer 54 alternate with writing / accumulating into the second frame buffer 54 and simultaneously reading from the first frame buffer 53.

[0131] FIG. 14 shows an example for the consecutive readout of S subframes within a HDR frame time tframe. The readout from the pixel counter units follows a rolling shutter scheme for pixel counter units 150 as illustrated in FIG. 12 and associated to pixel groups 25 as illustrated in FIG. 13.

[0132] The reset signals rst<0>, ..., rst<P-l> for the pixel rows 0, ..., P-1 are offset in time by a group readout period tgrp for transferring the pixel data from an entire pixel group to one of the frame buffers 53, 54.

[0133] Within each group readout period tgrp, the memory control unit 51 of FIG. 13 controls, for all pixel groups 25 simultaneously, the consecutive readout of the pixel data from the P pixel rows <0> to <P-1> to the memory row 21 and the transfer of the pixel data from the memory row 21 to one of the frame buffers 53, 54. In addition, the memory control unit 51 may assess the horizontal interrupts for the next pixel row.

[0134] The time for transferring the pixel data from one P pixel row to the memory row 21 and the transfer of the pixel data from the memory row 21 to one of the frame buffers 53, 54 is the row read time trow.

[0135] Within the row read time trow, “wr” indicates the time for the transfer of the pixel data to the memory row 21 and rd indicates the time for the transfer of the pixel data from the i-th column to frame buffer.

[0136] During the transfer of the pixel data from pixel row to the memory row 21, the row select signal sel is active. The trailing edge of the row select signal sel may also reset the horizontal interrupt.

[0137] FIG. 15 shows simultaneous reset signals rst<0>, ..., rst<P-l> for all pixel rows when the image sensor assembly 70 operates in a global shutter scheme. If the reset is asynchronous, counting during reset does typically not increase the counter value.

[0138] The readout of an entire subframe includes a rolling readout row by row. Since the exposure times would differ by the time it takes to read out the subframe, the first pixel row collects less light than the last pixel row. For the subframes 0 to S-2, this does not matter since any difference will spill over to the next subframe. Setting all digital auxiliary signals dO, dl, ... to inactive during the readout of the last subframe S-l allows the last subframe S-l to collect the spill overs.

[0139] FIG. 16 is a diagram illustrating an example in which the image sensor assembly 70 of FIG. 9 or FIG. 13 includes a CIS (stacked contact image sensor) having a two-layer structure with a radiation receiving chip 910 and a processing chip 920. The radiation receiving chip 910 may be a stacked substrate with singlephoton avalanche diodes formed in a first substrate and the pixel counter units formed in a second substrate. The processing chip 920 includes the further elements of the pixel circuits 100, e.g. the pixel memories and73954

[0140] 17

[0141] the digital core. As illustrated on the right side of FIG. 16, the image sensor assembly 70 is formed as one sensor by bonding the radiation receiving chip 910 and the processing chip 920 while electrically bringing contact pads on the radiation receiving chip 910 in contact with corresponding contact pads on the processing chip 920.

[0142] FIG. 17 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a system to which the technology according to an embodiment of the present disclosure can be applied.

[0143] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In the example depicted in FIG. 17, the vehicle control system 12000 includes a driving system control unit 12010, a body system control unit 12020, an outside-vehicle information detecting unit 12030, an in-vehicle information detecting unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound / image output section 12052, and a vehiclemounted network interface 12053 are illustrated as a functional configuration of the integrated control unit 12050.

[0144] The driving system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the driving system control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like.

[0145] The body system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle.

[0146] The outside-vehicle information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000. The outside-vehicle information detecting unit 12030 can be connected with an imaging section 12031. The outside-vehicle information detecting unit 12030 makes the imaging section 12031 imaging an image of the outside of the vehicle and receives the imaged image. Based on the received image, the outside-vehicle information detecting unit 12030 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto.73954

[0147] 18

[0148] The imaging section 12031 may be or may include an image sensor assembly according to the embodiments of the present disclosure. The light received by the imaging section 12031 may contain visible light and / or invisible light such as infrared rays or the like.

[0149] The in-vehicle information detecting unit 12040 detects information about the inside of the vehicle and may be or may include an image sensor assembly according to the embodiments of the present disclosure. The in-vehicle information detecting unit 12040 is, for example, connected with a driver state detecting section 12041 that detects the state of a driver. The driver state detecting section 12041, for example, includes a camera that includes the image sensor assembly according to the embodiments and that is focused on the driver. Based on detection information input from the driver state detecting section 12041, the in-vehicle information detecting unit 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver or may determine whether the driver is dozing.

[0150] The microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device based on the information about the inside or outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040 and output a control command to the driving system control unit 12010. For example, the microcomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like.

[0151] In addition, the microcomputer 12051 can perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040.

[0152] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information about the outside of the vehicle which information is obtained by the outsidevehicle information detecting unit 12030. For example, the microcomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detecting unit 12030.

[0153] The sound / image output section 12052 transmits an output signal of at least one of a sound or an image to an output device capable of visually or audible notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of FIG. 17, an audio speaker 12061, a display section 12062, and an instrument panel 12063 are illustrated as the output device. The display section 12062 may, for example, include at least one of an on-board display or a head-up display.73954

[0154] 19

[0155] FIG. 18 is a diagram depicting an example of the installation position of the imaging section 12031, wherein the imaging section 12031 may include imaging sections 12101, 12102, 12103, 12104, and 12105.

[0156] The imaging sections 12101, 12102, 12103, 12104, and 12105 are, for example, disposed at positions on a front nose, side-view mirrors, a rear bumper, and a back door of the vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle. The imaging section 12101 provided to the front nose and the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle 12100. The imaging sections 12102 and 12103 provided to the side view mirrors obtain mainly an image of the sides of the vehicle 12100. The imaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle 12100. The imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.

[0157] Incidentally, FIG. 18 depicts an example of photographing ranges of the imaging sections 12101 to 12104. An imaging range 12111 represents the imaging range of the imaging section 12101 provided to the front nose. Imaging ranges 12112 and 12113 respectively represent the imaging ranges of the imaging sections 12102 and 12103 provided to the side view mirrors. An imaging range 12114 represents the imaging range of the imaging section 12104 provided to the rear bumper or the back door. A bird's-eye image of the vehicle 12100 as viewed from above is obtained by superimposing image data imaged by the imaging sections 12101 to 12104, for example.

[0158] At least one of the imaging sections 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, imaging element having pixels for phase difference detection or may include a ToF module including an image sensor assembly according to the embodiments of the present disclosure.

[0159] For example, the microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect to the vehicle 12100 on the basis of the distance information obtained from the imaging sections 12101 to 12104, and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of the vehicle 12100 and which travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, equal to or more than 0 km / hour). Further, the microcomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automatic driving that makes the vehicle travel autonomously without depending on the operation of the driver or the like.

[0160] For example, the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance73954

[0161] 20

[0162] information obtained from the imaging sections 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that the driver of the vehicle 12100 can recognize visually and obstacles that are difficult for the driver of the vehicle 12100 to recognize visually. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle. In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062 and performs forced deceleration or avoidance steering via the driving system control unit 12010. The microcomputer 12051 can thereby assist in driving to avoid collision.

[0163] At least one of the imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 can, for example, recognize a pedestrian by determining whether there is a pedestrian in imaged images of the imaging sections 12101 to 12104. Such recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of the imaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When the microcomputer 12051 determines that there is a pedestrian in the imaged images of the imaging sections 12101 to 12104, and thus recognizes the pedestrian, the sound / image output section 12052 controls the display section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian. The sound / image output section 12052 may also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position.

[0164] The example of the vehicle control system to which the technology according to an embodiment of the present disclosure is applicable has been described above. By applying an image sensor assembly according to the embodiments of the present disclosure, the system can be improved. In particular, energy consumption can be reduced.

[0165] Additionally, embodiments of the present technology are not limited to the above-described embodiments, but various changes can be made within the scope of the present technology without departing from the gist of the present technology.

[0166] The image sensor assembly according to the present disclosure may be any device used for analyzing and / or processing radiation such as visible light, infrared light, ultraviolet light, and X-rays. For example, an image sensor assembly according to the embodiments may be any electronic device in the field of traffic, the field of home appliances, the field of medical and healthcare, the field of security, the field of beauty, the field of sports, the field of agriculture, the field of image reproduction or the like.

[0167] Specifically, in the field of image reproduction, the image sensor assembly according to the embodiments may be a device for capturing an image to be provided for appreciation, such as a digital camera, a smart phone, or a mobile phone device having a camera function. In the field of traffic, for example, a solid-state73954

[0168] 21

[0169] imaging device including an image sensor assembly according to the embodiments may be integrated in an in-vehicle sensor that captures the front, rear, peripheries, an interior of the vehicle, etc. for safe driving such as automatic stop, recognition of a state of a driver, or the like, in a monitoring camera that monitors traveling vehicles and roads, or in a distance measuring sensor that measures a distance between vehicles or the like.

[0170] In the field of home appliances, a solid-state imaging device with an image sensor assembly according to the embodiments may be integrated in any type of sensor that can be used in devices provided for home appliances such as TV receivers, refrigerators, and air conditioners to capture gestures of users and perform device operations according to the gestures. Accordingly, the solid-state imaging device with an image sensor assembly according to the embodiments may be integrated in home appliances such as TV receivers, refrigerators, and air conditioners and / or in devices controlling the home appliances. Furthermore, in the field of medical and healthcare, the solid-state imaging device with an image sensor assembly according to the embodiments may be integrated in any type of sensor, e.g., a camera device, provided for use in medical and healthcare, such as an endoscope or a device that performs angiography by receiving infrared light.

[0171] In the field of security, the solid-state imaging device with an image sensor assembly according to the embodiments can be integrated in a device provided for use in security, such as a monitoring camera for crime prevention or a camera for person authentication use. Furthermore, in the field of beauty, a solid-state imaging device with an image sensor assembly according to the embodiments can be used in a device provided for use in beauty, such as a skin measuring instrument that captures skin or a microscope that captures a probe. In the field of sports, a solid-state imaging device with an image sensor assembly according to the embodiments can be integrated in a device provided for use in sports, such as an action camera or a wearable camera for sport use or the like. Furthermore, in the field of agriculture, the solid-state imaging device with an image sensor assembly according to the embodiments can be used in a device provided for use in agriculture, such as a camera for monitoring the condition of fields and crops.

[0172] The present technology can also be configured as described below: [1] A pixel circuit (100), including: a single photon detector (110) configured to output count pulses in response to photons detected when enabled; a detector enable circuit (120) configured to enable the single photon detector (110) when a detector enable signal is active and to disable the single photon detector (110) otherwise; a counter circuit (130) configured to count the count pulses; and a control circuit (140) configured to control a duty cycle of the detector enable signal depending on a number of pulses counted in the counter circuit (130).

[0173] [2] The pixel circuit according to [1], wherein the control circuit (140) is configured to control a step-wise decrease of the duty cycle of the detector enable signal with increasing number of counts.

[0174] [3] The pixel circuit according to any of [1] and [2], wherein the single photon detector (110) includes a single-photon avalanche diode (111) and a quenching unit (115) including a quench path electrically connected in series with the single-photon avalanche diode (111).73954

[0175] 22

[0176] [4] The pixel circuit according to [3], wherein the detector enable circuit (120) includes an electronic switch (125) configured to reduce a voltage across the single-photon avalanche diode (111) when the detector enable signal is inactive.

[0177] [5] The pixel circuit according to any of [1] to [4], wherein the control circuit (140) is configured to connect an input of the detector enable circuit (120) with one of at least two global routing lines (241) in a first phase and with another one of the at least two global routing lines (241) in a second phase.

[0178] [6] The pixel circuit according to any of [1] to [5], wherein the control circuit (140) includes 2An routing switches (141) configured to be controlled by output signals of counter stages for the n most significant bits of the counter circuit (130), wherein each routing switch (141) is configured to connect an input of the detector enable circuit (120) with one of 2An global routing lines (241), with n being a positive integer number.

[0179] [7] The pixel circuit according to [6], wherein the control circuit (140) includes a n-to-2An line decoder (149) configured to control the 2An routing switches (141) in response to the output signals of the counter stages for the n most significant bits.

[0180] [8] The pixel circuit according to any of [6] and [7], wherein the global routing lines (241) are configured to transmit digital auxiliary signals, and wherein at least one of the digital auxiliary signals is a periodic signal with a period length equal to or shorter than a shortest exposure period.

[0181] [9] The pixel circuit according to [8], wherein at least some of the digital auxiliary signals contain periodic pulses of equal pulse length, and wherein for different digital auxiliary signals the pulses have different repetition rates.

[0182]

[0010] The pixel circuit according to any of [1] to [9], further including: a pixel memory (170) configured to receive and store the number of pulses counted in the counter circuit (130).

[0183]

[0011] An image sensor assembly (70), including: the pixel circuit (100) according to any of [5] to [9]; and a signal source (31) configured to generate digital auxiliary signals dO, dl, ... with different duty cycles and to pass each of the digital auxiliary signals dO, dl, ... to a different one of the global routing lines (241).

[0184]

[0012] The image sensor assembly according to

[0011] , including: a pixel array (10) including a plurality of pixel rows (11), each pixel row (11) including a plurality of pixel counter units (150) of the pixel circuits (100), wherein each pixel counter unit (150) includes at least the counter circuit (130) of the pixel circuit (170); and

[0185] a memory array (20) including pixel memories (170) configured to receive and store the number of pulses counted in the counter circuits (130).

[0186]

[0013] The image sensor assembly according to

[0012] , wherein the memory array (20) includes a pixel memory (170) for each of the pixel circuits (100).73954

[0187] 23

[0188]

[0014] The image sensor assembly according to any of

[0012] and

[0013] , including: a row controller (30) configured to control a transfer of the number of counts counted in the pixel circuits (100) of a selected pixel row (11) to the pixel memories (170) associated with the pixel circuits (100) of the selected pixel row (11) in a memory write period following the exposure period for the pixel row (11).

[0189]

[0015] The image sensor assembly according to

[0014] , wherein the signal source (31) is further configured to generate a first digital auxiliary signal dO being active for the exposure period and inactive for the write period.

[0190]

[0016] The image sensor assembly according to any of

[0014] and

[0015] , wherein the pixel circuits (100) of each pixel row (11) are configured to set a horizontal interrupt signal to an active level, when the number of counts counted in the counter circuit (130) exceeds a predefined threshold value.

[0191]

[0017] The image sensor assembly according to

[0016] , further including: a memory control unit (51) configured to control a readout of a pixel row (11) only when the horizontal interrupt signal hi for the pixel row (11) is active.

[0192]

[0018] The image sensor assembly according to any of

[0012] to

[0017] , wherein the pixel array (10) includes a number Q of pixel groups (15), each pixel group (15) including a number P of pixel rows (11), each pixel row (11) including a number N of pixel counter units (150), and wherein the memory array (20) includes Q memory rows (21) of the pixel memories (170).

[0193]

[0019] The image sensor assembly according to any of

[0012] to

[0018] , further including: a linearization unit (52) configured to obtain compensated count values from the count values stored in the memory array (20), wherein for the compensated count values each count step indicates a same amount of change of received light intensity.

[0194]

[0020] The image sensor assembly according to any of

[0012] to

[0019] , further including: a first frame buffer (53) configured to accumulate pixel data; a second frame buffer (54) configured to accumulate pixel data; and a frame controller (55) configured to control the first frame buffer (53) to accumulate first subframes, each first subframe including pixel data from the pixel counter units (150) of the pixel array (10) obtained during a first exposure period; and to control the second frame buffer (54) to accumulate second subframes, each second subframe including pixel data from the pixel counter units (150) of the pixel array (10) obtained during a second exposure period.

Claims

7395424CLAIMS1. A pixel circuit, comprising:a single photon detector configured to output count pulses in response to photons detected when enabled;a detector enable circuit configured to enable the single photon detector when a detector enable signal is active and to disable the single photon detector otherwise;a counter circuit configured to count the count pulses; anda control circuit configured to control a duty cycle of the detector enable signal depending on a number of pulses counted in the counter circuit.

2. The pixel circuit according to claim 1,wherein the control circuit is configured to control a step-wise decrease of the duty cycle of the detector enable signal with increasing number of counts.

3. The pixel circuit according to claim 1,wherein the single photon detector comprises a single-photon avalanche diode and a quenching unit comprising a quench path electrically connected in series with the single-photon avalanche diode.

4. The pixel circuit according to claim 3,wherein the detector enable circuit comprises an electronic switch configured to reduce a voltage across the single-photon avalanche diode when the detector enable signal is inactive.

5. The pixel circuit according to claim 1,wherein the control circuit is configured to connect an input of the detector enable circuit with one of at least two global routing lines in a first phase and with another one of the at least two global routing lines in a second phase.

6. The pixel circuit according to claim 1,wherein the control circuit comprises 2An routing switches configured to be controlled by output signals of counter stages for the n most significant bits of the counter circuit, wherein each routing switch is configured to connect an input of the detector enable circuit with one of 2An global routing lines, with n being a positive integer number.

7. The pixel circuit according to claim 6,wherein the control circuit comprises a n-to-2An line decoder configured to control the 2An routing switches in response to the output signals of the counter stages for the n most significant bits.

8. The pixel circuit according to claim 6,7395425wherein the global routing lines are configured to transmit digital auxiliary signals, and wherein at least one of the digital auxiliary signals is a periodic signal with a period length equal to or shorter than a shortest exposure period.

9. The pixel circuit according to claim 8,wherein at least some of the digital auxiliary signals contain periodic pulses of equal pulse length, and wherein for different digital auxiliary signals the pulses have different repetition rates.

10. The pixel circuit according to claim 1, further comprising:a pixel memory configured to receive and store the number of pulses counted in the counter circuit.

11. An image sensor assembly, comprising:the pixel circuit according to claim 5; anda signal source configured to generate digital auxiliary signals dO, dl, ... with different duty cycles and to pass each of the digital auxiliary signals dO, dl, ... to a different one of the global routing lines.

12. The image sensor assembly according to claim 11, comprising:a pixel array comprising a plurality of pixel rows, each pixel row comprising a plurality of pixel counter units of the pixel circuits, wherein each pixel counter unit comprises at least the counter circuit of the pixel circuit; anda memory array comprising pixel memories configured to receive and store the number of pulses counted in the counter circuits.

13. The image sensor assembly according to claim 12,wherein the memory array comprises a pixel memory for each of the pixel circuits.

14. The image sensor assembly according to claim 12, comprising:a row controller configured to control a transfer of the number of counts counted in the pixel circuits of a selected pixel row to the pixel memories associated with the pixel circuits of the selected pixel row in a memory write period following the exposure period for the pixel row.

15. The image sensor assembly according to claim 14,wherein the signal source is further configured to generate a first digital auxiliary signal dO being active for the exposure period and inactive for the write period.

16. The image sensor assembly according to claim 14,wherein the pixel circuits of each pixel row are configured to set a horizontal interrupt signal to an active level, when the number of counts counted in the counter circuit exceeds a predefined threshold value.

17. The image sensor assembly according to claim 16, further comprising:a memory control unit configured to control a readout of a pixel row only when the horizontal interrupt signal hi for the pixel row is active.

18. The image sensor assembly according to claim 12,wherein the pixel array comprises a number Q of pixel groups, each pixel group comprising a number P of pixel rows, each pixel row comprising a number N of pixel counter units, and wherein the memory array comprises Q memory rows of the pixel memories.

19. The image sensor assembly according to claim 12, further comprising:a linearization unit configured to obtain compensated count values from the count values stored in the memory array, wherein for the compensated count values each count step indicates a same amount of change of received light intensity.

20. The image sensor assembly according to claim 12, further comprising:a first frame buffer configured to accumulate pixel data;a second frame buffer configured to accumulate pixel data; anda frame controller configured to control the first frame buffer to accumulate first subframes, each first subframe comprising pixel data from the pixel counter units of the pixel array obtained during a first exposure period; and to control the second frame buffer to accumulate second subframes, each second subframe comprising pixel data from the pixel counter units of the pixel array obtained during a second exposure period.

Images