Photon counting circuit and photon counting method
The photon counting circuit optimizes counter usage by switching between modes based on signal overlap, reducing silicon area and enhancing imaging efficiency by addressing the challenges of high bit depth and simultaneous event detection in photon counting sensors.
Patent Information
- Application Number
- JP2025515698
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-09-22
- Filing Date
- 2023-09-19
- Publication Date
- 2025-09-19
AI Technical Summary
Existing photon counting sensors require high bit depth counters, occupying significant silicon area, making it difficult to implement small pixel pitches, and struggle with identifying simultaneous photon events, leading to reduced signal-to-noise ratios and power efficiency.
A photon counting circuit that determines temporal overlap between pixel signals to switch between coincidence and non-coincidence counting modes, using a common counter for simultaneous signals and individual counters for non-simultaneous signals, optimizing counter bit length and reducing silicon area.
This approach reduces the required counter bit length, saving silicon area and improving signal-to-noise ratios by efficiently handling simultaneous photon events, enabling smaller pixel designs and enhanced imaging capabilities.
Smart Images
Figure 2025531190000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure generally relates to photon counting circuits and methods. [Background technology]
[0002] It is commonly known to determine distance based on the round-trip delay of emitted light. For example, pulsed or modulated light can be emitted and the time it takes to reach an image sensor can be measured. Such a technique is sometimes called direct time-of-flight (dToF).
[0003] In dToF, photons are "counted" by time-to-digital conversion and stored in a histogram. Photon detection can be achieved by detecting photoelectron avalanches, for example, based on SPAD (single photon avalanche diode) technology, APD (avalanche photodiode), etc. The photoelectron signal either indicates the number of photons or the number of single photons, which may be stored in each counter, and may require a sufficiently high bit length to store a sufficient amount of photons.
[0004] Furthermore, photon counting techniques are known, for example based on photodiodes, where a determined voltage or the like indicates the number of photons.
[0005] Although techniques exist for counting photons, it is generally desirable to provide photon counting circuits and methods. Summary of the Invention [Means for solving the problem]
[0006] According to a first aspect, the present disclosure provides a photon counting circuit configured to determine whether signal simultaneity exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor, and if signal simultaneity exists, counts of the at least two pixel signals are counted together in the coincidence counting operation mode to set a counting operation mode to the coincidence counting operation mode, and generate a coincidence counting signal to set the counting operation mode to the coincidence counting operation mode.
[0007] According to a second aspect, the present disclosure provides a photon counting method, including determining whether signal simultaneity exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor, and if signal simultaneity exists, counts of the at least two pixel signals are counted together in a coincidence counting operation mode to set a counting operation mode to a coincidence counting operation mode, and generating a coincidence counting signal to set the counting operation mode to the coincidence counting operation mode.
[0008] Further aspects are set out in the dependent claims, the following description and the drawings.
[0009] Embodiments will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]
[0010] [Figure 1] 1 shows a schematic diagram of a SPAD-based photon-counting pixel according to the present disclosure. [Figure 2] 1 illustrates, in block diagram form, one embodiment of a photonic circuit including a photon counting circuit according to the present disclosure. [Figure 3] 10 illustrates a further embodiment of a photonic circuit including a photon counting circuit according to the present disclosure. [Figure 4] 4 shows a timing diagram of the photon counting circuit of FIG. 3. [Figure 5] 1 illustrates a photon-counting ToF circuit and a photon-counting circuit according to the present disclosure operating in non-coincidence mode. [Figure 6] 1 illustrates a photon-counting ToF circuit and a photon-counting circuit according to the present disclosure operating in coincidence mode. [Figure 7] 1 illustrates one embodiment of a photon counting method according to the present disclosure in which it is determined whether coincidence or non-coincidence counts exist. [Figure 8] 10 illustrates a further embodiment of a photon counting method according to the present disclosure in which non-coincidence is determined. [Figure 9] 10 illustrates a further embodiment of a photon counting method according to the present disclosure in which the presence of coincidence is determined. [Figure 10] 10 illustrates a further embodiment of a photon counting method according to the present disclosure, in which each counter is configured as a common counter or an individual counter depending on whether coincidence exists. [Figure 11] 1 shows a high-level diagram of a photon-counting ToF camera including photon-counting circuitry according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0011] Before providing a detailed description of the embodiment starting with FIG. 1, a general description will be provided.
[0012] As mentioned in the introduction, counting photons to determine distance or depth is commonly known. However, to increase the dynamic range (DR), as many counter bits as possible may be required. It has been recognized that it may be desirable to increase the effective bit length of the counter while simultaneously reducing the silicon area required, thereby enabling a smaller pitch in pixel designs.
[0013] It has further been recognized that there may be temporal (in the time domain) coherence / simultaneity between / among pixels (adjacent / neighboring, the present disclosure is not limited in this respect) of a photon-counting ToF sensor that can be used to reduce the counter length. It should be noted that the present disclosure is not limited to active light sensing as known for ToF, and photons from passive light (e.g., ambient light, sunlight, etc.) can also be counted based on other sensor types.
[0014] Furthermore, it has been recognized that it may be desirable to provide a SPAD / APD-based photon-counting sensor that can be used in a binning mode. It has been recognized that binning is difficult to achieve in conventional sensors without identifying simultaneously generated events. Without an identification mechanism, known sensors treat all simultaneously generated events as a single event. However, this can result in a reduction in the total effective counts, potentially reducing signal-to-noise ratios (SNRs) and power efficiency.
[0015] Accordingly, some embodiments relate to a photon counting circuit configured to determine whether signal synchrony exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor, and if signal synchrony exists, counts of the at least two pixel signals are counted together in the coincidence counting operation mode to set the counting operation mode to the coincidence counting operation mode, and generate a coincidence counting signal to set the counting operation mode to the coincidence counting operation mode.
[0016] The circuitry may relate to any entity or multiple entities usable in the context of photon counting (time-of-flight) measurements, such as one or more processors (e.g., CPUs (Central Processing Units), GPUs (Graphics Processing Units)), one or more FPGAs (Field Programmable Gate Arrays), etc. The circuitry may include or be applied to (a system of) camera(s), (a system of) computer(s), server(s), etc.
[0017] The photon counting circuit may be used to count photons based on a photoelectron signal that may be generated by one or more SPADs (single photon avalanche diodes), APDs (avalanche photodiodes), or any other circuit that can generate a signal indicative of one or more photons so that the number of incident photons may be derived.
[0018] In general, the present disclosure is not limited to photon-counting ToF technology, as 2D (two-dimensional) active or passive imaging can also be applied with the principles of the present disclosure. For example, the present disclosure can generally be applied based on photon counting, i.e., any type of technology that can count photons apart from ToF. Thus, in some embodiments, the present disclosure can generally relate to photon-counting circuits, photodetection circuits, 2D imaging circuits, 3D imaging circuits, etc.
[0019] Also, a SPAD or the like can be used in a pixel of a photon-counting time-of-flight sensor according to the present disclosure for light detection or light counting. When a photon is incident on a pixel, the pixel can generate a pixel signal that overlaps in time with another pixel signal of another pixel of the photon-counting ToF sensor. The sensor can be any type of sensor, such as a single-layer sensor, a stacked sensor, a BSI (back-side illuminated) sensor, or an FSI (front-side illuminated) sensor.
[0020] Temporal overlap may refer to when at least two pixel signals are generated at approximately the same time. If they are displayed at exactly the same time (and, in some embodiments, they also end at the same time), their degree of overlap may be determined to be 100 percent (or any other way of expressing that they are generated completely in parallel). The respective internal times for determining the degree of temporal overlap may vary from pixel to pixel and may be calibrated such that, globally, at least two pixel signals may be determined to overlap to some degree.
[0021] The degree may be less than 100 percent (or "perfect") if they are not generated exactly simultaneously, ie, if at least two pixel signals are shifted in time relative to each other.
[0022] For example, they may not overlap at all, or may overlap to a certain extent (e.g., 50 percent). For example, assuming the pixel signals have the same length, if a first pixel signal is generated at a first time point and a second pixel signal is generated at a second time point that is halfway between the first time points, the overlap may be 50 percent.
[0023] If the degree of temporal overlap is sufficiently high (i.e., equal to or greater than a predetermined threshold), it can be determined that the signals are generated simultaneously, in other words, that there is signal simultaneity between at least two pixel signals. Of course, in the case of more than two pixel signals, it can be determined whether or not there is simultaneity in a similar or identical manner.
[0024] When signal synchronism exists, a coincidence operation mode can be applied in accordance with the present disclosure. In the coincidence operation mode, the counts indicated by each of at least two pixel signals are counted together (e.g., counted as one). For example, when synchronism exists, the respective counts can be counted as one in a common counter that can be common to at least two pixels.
[0025] Apart from the coincidence counting operation mode, a non-coincidence counting operation mode may be determined as the counting operation mode when there is no signal simultaneity of at least two pixel signals, in which case each count is stored in a separate counter of at least two pixels.
[0026] To indicate or set the counting mode of operation to a coincidence mode of operation, a coincidence signal may be generated, which corresponds to one of the at least two pixel signals and may be reused or forwarded to a respective circuit for "activating" a common counter (or other circuit for storing counts, such as a histogram).
[0027] When indicating or setting the non-coincidence counting operation mode as the counting operation mode, it may not be necessary to generate a respective signal, since, like the coincidence counting operation mode, the non-coincidence counting operation mode can be explicitly set with a signal and, without limiting the disclosure in this respect, the non-coincidence counting operation mode can be determined based on the absence of a signal.
[0028] In some embodiments, a non-coincidence mode of operation may be indicated by a respective signal, and a coincidence mode of operation may be determined based on the absence of the signal.
[0029] The present disclosure can save silicon area. Prior art photon counting sensors may require a counter with a sufficiently high bit depth to store all generated events. However, such prior art counters may occupy too much silicon area, making it difficult to implement small pixel pitch designs.
[0030] However, in accordance with this disclosure, it is recognized that it may be desirable to consider correlation between adjacent / nearby pixels within the sensor.
[0031] This correlation is utilized to save the total counter bit length of two or more pixels in a pixel group (e.g., 1x2, 2x1, 2x2, etc.). Thus, in accordance with the present disclosure, this temporal correlation (i.e., coincidence) can be detected / determined and the coincidence counts can be stored in a common (shared) counter. In this way, unlike counting with individual counters for each pixel, the coincidence counts do not need to be stored multiple times in each individual counter, but only once in the common counter. Thus, the total counter bit length of the pixel group can be reduced, thereby saving silicon area.
[0032] In some embodiments, the photon counting circuitry is further configured to determine the counting operation mode as a non-coincidence counting operation mode when the degree of temporal overlap of at least two pixel signals is less than a predetermined threshold, as described herein.
[0033] In some embodiments, the photon counting circuitry is further configured to store photon counts indicated by the at least two pixel signals in a common counter that is common to the at least two pixels in a coincidence counting mode of operation, as described herein.
[0034] In some embodiments, the photon counting circuitry is further configured to store one simultaneous photon count of the at least two pixel signals as one count in a common counter in a coincidence counting mode of operation as described herein.
[0035] In some embodiments, the photon counting circuitry is further configured to store photon counts indicated by the at least two pixel signals in a separate counter for each of the at least two pixel signals in a non-coincidence mode of operation as described herein.
[0036] In some embodiments, the coincidence signal is a clock signal. However, as described above, the coincidence signal may be one of at least two pixel signals that can be reused (or may be a clock signal). Furthermore, the clock signal may be generated in response to receiving one of the at least two pixel signals.
[0037] In some embodiments, the coincidence signal corresponds to a pixel signal that is later in time of the at least two pixel signals, whereby it may take some time to determine whether coincidence exists, and therefore it may be ensured that coincidence exists first before controlling the respective counters.
[0038] In some embodiments, the photon counting further includes at least three counters, including at least one common counter for a coincidence counting mode of operation and at least two individual counters for a non-coincidence counting mode of operation, as described below (e.g., with reference to FIG. 2, FIG. 5, or FIG. 6).
[0039] In some embodiments, the photon counting circuit further includes four counters, including first through fourth counters, and the circuit may be further configured to set the first and second counters as a common counter in a coincidence counting mode of operation based on the coincidence counting signal, as described below (e.g., with reference to Figures 5 and 6).
[0040] In some embodiments, the photon counting circuit is further configured to set the first and third counters as first individual counters and set the second and fourth counters as second individual counters based on the absence of a coincidence signal, as described below (e.g., with reference to Figures 5 and 6).
[0041] Typically, by configuring each counter as described above, a dynamically adaptable counting circuit may be provided that may be able to dynamically switch between common and individual counting depending on whether there is simultaneity or not. This may optimize storage, for example, by having two (smaller) counters function as one large counter that stores coincidence counts, thereby reducing the amount of storage required (when two counts are stored together as one) and increasing the amount of storage possible. When asynchronous counts are present, the common counters may be dynamically configured to be individual counters, thereby optimizing spatial resolution for the asynchronous case.
[0042] Some embodiments relate to a photon counting method, as described herein, that includes determining whether signal synchrony exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor, and if signal synchrony exists, counts of the at least two pixel signals are counted together in a coincidence counting mode of operation to set the counting mode of operation to a coincidence counting mode of operation, and generating a coincidence counting signal to set the counting mode of operation to the coincidence counting mode of operation.
[0043] The methods described herein can be implemented by photon-counting circuits according to the present disclosure.
[0044] In some embodiments, the photon counting method further includes determining the counting mode of operation as a non-coincidence mode of operation when the degree of temporal overlap of the at least two pixel signals is equal to or less than a predetermined threshold, as described herein. In some embodiments, the photon counting method further includes, in the coincidence mode of operation, storing photon counts indicated by the at least two pixel signals in a common counter common to the at least two pixels, as described herein. In some embodiments, the photon counting method further includes, in the coincidence mode of operation, storing one coincident photon count of the at least two pixel signals as one count in the common counter, as described herein. In some embodiments, the photon counting method further includes, in the non-coincidence mode of operation, storing the photon counts indicated by the at least two pixel signals in separate counters for each of the at least two pixel signals, as described herein. In some embodiments, the coincidence signal is a clock signal, as described herein. In some embodiments, the coincidence signal corresponds to a temporally later of the at least two pixel signals, as described herein. In some embodiments, the photon counting method is applied using at least three counters, including at least one common counter for a coincidence mode of operation and at least two individual counters for a non-coincidence mode of operation, as described herein. In some embodiments, the photon counting method is applied using four counters, including first through fourth counters, and the method further includes configuring the first and second counters as common counters for the coincidence mode of operation based on a coincidence signal, as described herein. In some embodiments, the photon counting method further includes configuring the first and third counters as first individual counters and the second and fourth counters as second individual counters based on the absence of a coincidence signal, as described herein.
[0045] The methods described herein are also, in some embodiments, implemented as a computer program that, when executed on a computer and / or processor, causes the computer and / or processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided having stored thereon a computer program product that, when executed by a processor, such as the processors described above, causes the method described herein to be performed.
[0046] Returning to Figure 1, a schematic diagram of a SPAD-based photon-counting pixel 1 and PFE (Pixel Front Element (circuit)) is shown. In this embodiment, a SPAD 2 device biased in Geiger mode generates a pulse at its anode, i.e., a so-called "event." This pulse is filtered by a comparator / inverter (or any other interface circuit), and then the events are counted by a counter 3.
[0047] It should be noted that while two pixels are shown in FIG. 1 for illustrative purposes, the present disclosure may generally be applied to larger numbers of pixels.
[0048] In this embodiment, for simplicity, SPAD 2 is depicted as a passive quench structure, but it may also be an active quench SPAD, an external clock-controlled quench SPAD, etc. Furthermore, the present disclosure is not limited to SPADs but may also be based on one or more APDs or any other pixel with similar characteristics that generates digital events in response to incident photons. Counter 3 is simplified by a counter block, but it may also be a ripple counter, an LFSR counter, or any other circuit capable of accumulating events generated by a SPAD.
[0049] FIG. 2 illustrates in block diagram form one embodiment of a photon-counting ToF circuit 10 that includes a photon-counting circuit 11 embodied as a “coincidence detection” block 11 .
[0050] In this embodiment (without limiting the disclosure in this respect), two pixels (as already shown in FIG. 1) are shown, each based on a SPAD, namely SPAD1 and SPAD2. A coincidence detection block 11 is configured to monitor the outputs of clocks CLK1 and CLK2 of SPAD1 and SPAD2, respectively.
[0051] If the clock signals (corresponding to pixel signals as described above) generated by the two SPADs, i.e., CLK1 and CLK2, are simultaneous signals or have a sufficiently high overlap in the time domain, the coincidence detection block determines them as a simultaneous event and outputs YES (meaning a simultaneous event; any other way of expressing coincidence can be envisioned, and the present disclosure is not limited to outputting YES) to turn on switch SW2, thereby passing the generated simultaneous clock signal CLK through switch SW2 to the common counter 12. In this process, another switch SW1 is turned off (i.e., kept in a disconnected state).
[0052] If the clock signals CLK1 and CLK2 are not simultaneous signals, the coincidence detection block determines that it is a non-simultaneous event and outputs NO (meaning not a simultaneous event), thereby turning on switch SW1 and passing the two clock signals through switch SW1 to the individual counter 13 for SPAD1 and the individual counter 13' for SPAD2.
[0053] In some embodiments, some time may be required for the concurrency circuit to complete its decision, and buffered clock signals CLK1D and CLK2D are used, which are buffered versions of CLK1 and CLK2, respectively, caused by delay cells.
[0054] 3 and 4 show a further embodiment of a photon-counting ToF circuit 20 including a photon-counting circuit 21 according to the present disclosure (FIG. 3), and a clock diagram 30 (FIG. 4) to show how the photon-counting circuit 21 can be controlled.
[0055] 3, the outputs of two SPADs, CLK1 and CLK2, are monitored to generate a further clock signal CLK3. In this embodiment, photon counting circuit 21 is configured to detect overlap between clock signals CLK1 and CLK2 and to output CLK3 when overlap between CLK1 and CLK2 is detected.
[0056] 4a) and 4b) show embodiments of clock signals in the simultaneous case, and 4b) and 4c) show embodiments of clock signals in the non-simultaneous case.
[0057] As can be seen from a) and b), the clock signal CLK3 corresponds to the later of the clock signals CLK1 and CLK2, but CLK3 is only output when there is a sufficient degree of overlap between CLK1 and CLK2.
[0058] As shown in c) and d), when the two clock signals CLK1 and CLK2 do not overlap at all and there is no coincidence, the CLK3 signal is not generated.
[0059] According to the present disclosure, it may be possible to use photon counting circuitry in binned pixels (e.g., the two pixels of FIG. 3 , without limiting the present disclosure in this respect). For example, if two pixels in FIG. 3 are to be binned and generate simultaneous pixel signals as described herein (i.e., signals CLK1 and CLK2 overlap in time to some extent), only one count will be recorded if there is no coincidence detection. However, applying the same or similar circuitry as in FIG. 3 to such binned pixels can double the count of the common counter to reproduce the actual number of events. This may result in more counts being accumulated, optimizing the signal-to-noise ratio (typically equivalent to the square root of the count).
[0060] Further embodiments of photon-counting ToF circuit 40 including a photon-counting circuit are depicted in Figures 5 and 6, which depict the same circuit but in different states of operation. Figure 5 shows the "coincidence mode" as such deactivated, with counters 41 and 42 always operating together as separate counters for SPAD1, and counters 43 and 44 always operating together as separate counters for SPAD2. Thus, the circuit of Figure 5 operates as a differential-mode counter.
[0061] 6 depicts a state in which the "coincidence mode" is activated, which generally means that coincidence can be detected. In this state, counters 41 and 43 operate together as a common counter for SPAD1 and SPAD2 when coincidence is determined, and counters 42 and 44 are single counters when there is no coincidence. Thus, the circuit of FIG. 6 operates as a common-mode counter.
[0062] A photon counting circuit 45 is depicted separately from the counters 41-44. As already mentioned above, delay lines 46 and 47 are used to generate delayed clock signals in case of asynchronous determination. Furthermore, a multiplexer with a control signal "ModSel" (mode select) is added to switch the photon counting ToF circuit 40 between a normal mode with individual counters per pixel as shown in FIG. 5 and a shared counter mode as shown in FIG. 6. This allows the circuit to have the flexibility to switch between these two modes.
[0063] When ModSel is zero (as in FIG. 5), there are two separate counters based on counters 41 and 42 for SPAD1 and counters 43 and 44 for SPAD2. In this mode, each of the separate counters is multiplied by log2(2 M +2 N ), where M is the bit length of counters 42 and 44, and N is the bit length of counters 41 and 43, and the present disclosure is not limited to any particular bit length, and the bit lengths of counters 42 and 44 and counters 41 and 43 may differ from each other.
[0064] If ModSel is 1 (as in Figure 6), coincidence counting is used. When coincidence is detected, a signal in the form of a clock signal (not shown) is generated by the photon counting circuit 45, similar to that shown in Figure 4a) or b).
[0065] This clock pulse is counted by the common counter formed by counters 41 and 43, as already explained above. The common counter formed has twice the length of N bits. If no coincidence is detected, no signal is generated by photon counting circuit 45 and each pixel signal is counted separately by counters 42 and 44, which act as individual counters.
[0066] For example, if the counter bit length N is 7 and the counter bit length M is 5, the common counter has 14 bits, and therefore, the 14-bit (i.e., log2(2 14 +2 5 ) a common value and a 5-bit difference value can be stored, but in a mode where no coincidence counting is activated at all (FIG. 5), the total bit length is 12 bits.
[0067] FIG. 7 illustrates in block diagram form one embodiment of a photon counting method 50 according to the present disclosure.
[0068] At 51, it is determined whether there is a coincidence, as described herein.
[0069] At 52, once coincidence is determined, a coincidence signal is generated as described herein.
[0070] FIG. 8 illustrates in block diagram form one embodiment of a photon counting method 60 according to the present disclosure.
[0071] At 61, it is determined that non-simultaneity exists, as described herein.
[0072] At 62, the photon counts are stored in individual counters as described herein.
[0073] FIG. 9 illustrates in block diagram form one embodiment of a photon counting method 70 according to the present disclosure.
[0074] At 71, it is determined that coincidence exists as described herein.
[0075] At 72, a coincidence signal is generated as described herein.
[0076] At 73, the simultaneous photon counts are stored in a common counter as described herein.
[0077] FIG. 10 illustrates in block diagram form one embodiment of a photon counting method 75 according to the present disclosure.
[0078] At 76, it is determined whether there is a coincidence, as described herein.
[0079] At 77, each counter is set to a common counter or an individual counter depending on whether there is a coincidence or not, as described herein.
[0080] In FIG. 11, there is shown one embodiment of a high-level photon-counting time-of-flight imaging system 80, here embodied as a photon-counting ToF camera, which may be used for depth sensing or provide distance measurements, configured to carry out the methods as described herein and having photon counting circuitry 87 that forms control of the photon-counting ToF device 80 (and, although not shown, includes corresponding processor, memory, and storage (i.e., counters)).
[0081] The photon counting ToF imaging system 80 has a pulsed light source 81, which includes a light emitting element (based on a laser diode), which in this embodiment is a narrow band laser element.
[0082] A light source 81 emits pulses of light onto a scene 82 (area or object of interest) that reflects the light. By repeatedly emitting light onto the scene 82, the scene 82 can be scanned, as is commonly known to those skilled in the art. The reflected light is focused onto a photodetector 84 by an optical stack 83.
[0083] The photon counting time-of-flight photodetector circuit 87 also forms the control for the light source, as does the corresponding control circuitry (not shown).
[0084] The photodetector 84 comprises an image sensor 85 implemented based on a plurality of SPADs (single photon avalanche diodes) formed into an array of pixels (image sensors), and a microlens array 86 that focuses light reflected from the scene 82 onto the image sensor 85 (onto each pixel of the image sensor 85).
[0085] Emission time information is provided from the light source 81 to a photon counting circuit 87, which includes a photon counting unit 88, when light reflected from the scene 82 is detected, and which also receives respective time information from the image sensor 85. Generally, a photon counting time-of-flight system is also capable of performing time-of-flight measurements. Based on the emission time information received from the light source 81 and the arrival time information received from the image sensor 85, the photon counting unit 88 calculates the round-trip time of the light emitted from the light source 81 and reflected from the scene 82, and based thereon, calculates the distance d (depth information) between the image sensor 85 and the scene 82 based on the determination of a light event, as described herein. Furthermore, as described herein, the photon counting unit 88 has information regarding when common-mode counting and differential-mode counting are activated so that it can assign counts, respectively.
[0086] The depth information is supplied from the photon counting unit 88 to a 3D image reconstruction unit 89 of the photon counting circuit 87, which reconstructs (generates) a 3D image of the scene 82 based on the depth information received from the time-of-flight measurement unit 88.
[0087] The embodiments are intended to describe the method in an exemplary order of method steps. However, the specific order of method steps is provided for illustrative purposes only and is not intended to be binding. Modifications to the order of method steps will be apparent to those skilled in the art.
[0088] It should be noted that the division of control 87 into units 88 and 89 is for illustrative purposes only, and the present disclosure is not limited to any particular division of functionality in any particular unit. For example, control 87 may be implemented by a respective programmed processor, field programmable gate array (FPGA), etc.
[0089] The methods described herein may also be implemented as a computer program product that, when executed on a computer and / or processor, causes the computer and / or processor to perform the method. In some embodiments, a non-transitory computer-readable storage medium is also provided having embodying therein a computer program product that, when executed by a processor, such as the processors described above, causes the described method to be performed.
[0090] All units and entities described in this specification and claimed in the appended claims may, unless otherwise specified, be implemented as integrated circuit logic on a chip, for example, and the functions provided by such units and entities may, unless otherwise specified, be implemented by software.
[0091] To the extent that the embodiments of the present disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be understood that computer programs providing such software control, and the transmission, storage, or other media on which such computer programs are provided, are also contemplated as aspects of the present disclosure.
[0092] The present technology can also be configured as described below. (1) determining whether signal synchronism exists for at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor based on a degree of temporal overlap of the at least two pixel signals, and if signal synchronism exists, counts of the at least two pixel signals are counted together in the coincidence counting operation mode to set a counting operation mode to the coincidence counting operation mode; a photon-counting circuit configured to generate a coincidence signal to set the counting mode of operation to the coincidence mode of operation; (2) The photon counting circuit according to (1), further configured to determine the counting operation mode as a non-coincidence counting operation mode when the degree of temporal overlap of the at least two pixel signals is less than a predetermined threshold. (3) The photon counting circuit according to (1) or (2), further configured to store, in the coincidence operation mode, photon counts indicated by the at least two pixel signals in a common counter that is common to the at least two pixels. (4) The photon counting circuit according to (3), further configured to store a simultaneous photon count of one of the at least two pixel signals as one count in the common counter in the coincidence counting operation mode. (5) A photon counting circuit according to any one of (1) to (4), further configured to store, in the non-coincidence counting operation mode, photon counts indicated by the at least two pixel signals in separate counters for each of the at least two pixel signals. (6) A photon counting circuit according to any one of (1) to (5), wherein the coincidence signal is a clock signal. (7) The photon counting circuit according to (6), wherein the coincidence signal corresponds to the later pixel signal of the at least two pixel signals. (8) The photon counting circuit according to any one of (1) to (7), further comprising at least three counters including at least one common counter for the coincidence counting operation mode and at least two individual counters for the non-coincidence counting operation mode. (9) The photon counting circuit according to any one of (1) to (8), further comprising four counters including first to fourth counters, wherein the circuit is further configured to set the first and second counters as a common counter for the coincidence counting operation mode based on the coincidence counting signal. (10) The photon counting circuit according to (9), further configured to set the first and third counters as first individual counters and set the second and fourth counters as second individual counters based on the absence of the coincidence signal. (11) A photon counting method including: determining whether signal simultaneity exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor; and if signal simultaneity exists, counts of the at least two pixel signals are counted together in a coincidence counting operation mode to set the counting operation mode to the coincidence counting operation mode; and generating a coincidence counting signal to set the counting operation mode to the coincidence counting operation mode. (12) The photon counting method according to (11), further comprising a step of determining the counting operation mode as a non-coincidence counting operation mode when the degree of temporal overlap of the at least two pixel signals is equal to or less than a predetermined threshold. (13) The photon counting method according to (11) or (12), further comprising the step of storing, in the coincidence operation mode, photon counts indicated by the at least two pixel signals in a common counter common to the at least two pixels. (14) The photon counting method according to (13), further comprising the step of storing, in the coincidence operation mode, one simultaneous photon count of the at least two pixel signals in the common counter as one count. (15) A photon counting method according to any one of (11) to (14), further comprising the step of storing, in the non-coincidence counting operation mode, photon counts indicated by the at least two pixel signals in separate counters for each of the at least two pixel signals. (16) The photon counting method according to any one of (11) to (15), wherein the coincidence signal is a clock signal. (17) The photon counting method according to (16), wherein the coincidence signal corresponds to the later pixel signal of the at least two pixel signals. (18) The photon counting method according to any one of (11) to (17), applied using at least three counters including at least one common counter for the coincidence counting operation mode and at least two individual counters for the non-coincidence counting operation mode. (19) The photon counting method according to any one of (11) to (18), which is applied using four counters including first to fourth counters, and further comprises a step of setting the first and second counters as a common counter for the coincidence counting operation mode based on the coincidence counting signal. (20) The photon counting method according to (19), further comprising the step of setting the first and third counters as first individual counters and setting the second and fourth counters as second individual counters based on the absence of the coincidence signal. (21) A computer program comprising program code that, when executed on a computer, causes the computer to perform any one of the methods described in (11) to (20). (22) A non-transitory computer-readable recording medium storing a computer program product that, when executed by a processor, causes the method according to any one of (11) to (20) to be performed.
Claims
1. determining whether signal synchronism exists for at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor based on a degree of temporal overlap of the at least two pixel signals, and if signal synchronism exists, counts of the at least two pixel signals are counted together in the coincidence counting operation mode to set a counting operation mode to the coincidence counting operation mode; a photon counting circuit configured to generate a coincidence signal to set the counting operation mode to the coincidence operation mode;
2. 2. The photon counting circuit of claim 1, further configured to determine the counting operation mode as a non-coincidence counting operation mode when a degree of temporal overlap of the at least two pixel signals is less than a predetermined threshold.
3. 2. The photon counting circuit of claim 1, further configured to store photon counts indicated by the at least two pixel signals in a common counter that is common to the at least two pixels in the coincidence counting operation mode.
4. 4. The photon counting circuit of claim 3, further configured to store a coincident photon count of one of the at least two pixel signals as one count in the common counter in the coincidence counting operation mode.
5. 2. The photon counting circuit of claim 1, further configured to, in the non-coincidence operation mode, store photon counts indicated by the at least two pixel signals in separate counters for each of the at least two pixel signals.
6. 2. The photon counting circuit according to claim 1, wherein the coincidence signal is a clock signal.
7. 7. The photon counting circuit according to claim 6, wherein the coincidence signal corresponds to the later pixel signal of the at least two pixel signals.
8. 2. The photon counting circuit of claim 1, further comprising at least three counters including at least one common counter for said coincidence mode of operation and at least two individual counters for said non-coincidence mode of operation.
9. 2. The photon counting circuit of claim 1, further comprising four counters including first through fourth counters, wherein the circuit is further configured to set the first and second counters as a common counter for the coincidence counting mode of operation based on the coincidence counting signal.
10. 10. The photon counting circuit of claim 9, further configured to configure the first and third counters as first individual counters and configure the second and fourth counters as second individual counters based on the absence of the coincidence signal.
11. 1. A photon counting method comprising: determining whether signal synchronism exists for at least two pixel signals based on a degree of temporal overlap of at least two pixel signals of at least two pixels of a photon counting time-of-flight sensor; and if signal synchronism exists, counts of the at least two pixel signals are counted together in a coincidence counting operation mode to set the counting operation mode to the coincidence counting operation mode; and generating a coincidence counting signal to set the counting operation mode to the coincidence counting operation mode.
12. 12. The photon counting method of claim 11, further comprising determining the counting operation mode as a non-coincidence counting operation mode if the degree of temporal overlap of the at least two pixel signals is equal to or less than a predetermined threshold.
13. 12. The photon counting method of claim 11, further comprising the step of storing, in the coincidence operation mode, photon counts indicated by the at least two pixel signals in a common counter common to the at least two pixels.
14. 14. The photon counting method of claim 13, further comprising the step of storing, in the coincidence operation mode, a coincident photon count of one of the at least two pixel signals as one count in the common counter.
15. 12. The photon counting method of claim 11, further comprising the step of: in the non-coincidence operation mode, storing photon counts indicated by the at least two pixel signals in separate counters for each of the at least two pixel signals.
16. The photon counting method according to claim 11 , wherein the coincidence signal is a clock signal.
17. 17. The photon counting method according to claim 16, wherein the coincidence signal corresponds to the later pixel signal of the at least two pixel signals.
18. 12. The photon counting method of claim 11 applied with at least three counters, including at least one common counter for the coincidence mode of operation and at least two individual counters for the non-coincidence mode of operation.
19. 12. The photon counting method of claim 11, applied using four counters including first to fourth counters, and further comprising the step of setting the first and second counters as a common counter for the coincidence counting mode of operation based on the coincidence counting signal.
20. 20. The photon counting method of claim 19, further comprising the step of: based on the absence of the coincidence signal, setting the first and third counters as first individual counters and setting the second and fourth counters as second individual counters.