Pixel sensor system

Abstract

To provide a system of detecting events at very low photon budgets, which can, in a very short scanning time, determine whether the detection is a real detection or a false detection by ambient light or thermal noise.SOLUTION: A system comprises an array with a plurality of single-photon detectors. Row buses and column buses are coupled to an evaluation circuit. The evaluation circuit is configured to evaluate aggregated signals from the row buses and column buses against predetermined confirmation patterns to confirm a detection of an incident photon and location determination thereof, where the confirmation patterns comprise temporal and / or spatial conditions. A detection is confirmed on the basis of a compliance of row bus and column bus signals with a predetermined pattern.SELECTED DRAWING: Figure 4A

Description

[Technical Field]

[0001] The present invention relates to an improved system and method for high speed imaging that can reliably capture an environment over a limited time span and with a very limited photon budget. [Background technology]

[0002] In the prior art, active imaging is performed by moving a light beam, usually using a laser, over the area to be captured, and the position where the beam impinges is recorded at each point in time via multiple image sensors. By processing the difference in position from different viewpoints (sensors), the effective distance to the illuminated target can be determined by triangulation. Such measurements capture voxels. The speed at which this process can be performed, i.e., the voxel capture rate, is limited on the one hand by the scanning speed of the light beam, but also (most strongly) by the processing time required for the sensor to detect the reflected light beam, especially in relation to background radiation (extraneous light) and general thermal noise. Addressing this second issue in particular can significantly speed up imaging.

[0003] To achieve voxel capture rates of tens or hundreds of millions of voxels per second, each voxel must be captured within a time span of 10 ns or less. As a result, the sensor must also be suitable for operating with a limited photon budget (i.e., the number of photons that can be incident on the sensor and detected that are sufficient for "detection"), such as 10 photons. With a limited processing time span, only a limited number of photons can be captured.

[0004] Existing image processing and imaging systems process the optical input acquired by the sensor either in parallel for all pixels, in the case of a so-called "global shutter," or in a time-distributed manner, in the case of a "rolling shutter." In either case, to obtain a signal that can be recorded above the minimum detection voltage, typical imaging systems have a gain factor of 10 μV to 1 mV per incident electron. The upper limit of this range is guaranteed only by very recent imagers, which are specifically modified to count photons and focus on a very low detection rate. Using such specialized sensors, a 10 mV signal can be generated for 10 electrons, resulting in a "positive" reading for a particular pixel. To detect an incoming photon packet, at least 10 photons must strike the sensor within a 10 ns time span. Previous prior art approaches have limited the sensor's exposure time to, say, 10 ns, and then read the sensor to trigger an event via a threshold voltage (i.e., the sensor reads the true incidence of the reflected beam, not false positives due to ambient light or thermal noise). The drawback of this is that the time consumed to read the sensor dominates the process, especially for high-resolution imagers, and is typically significantly larger than 10 ns, creating a bottleneck here.

[0005] Alternatively, instead of reading the sensor and determining whether an event has occurred based on the data read, the sensor itself determines whether a minimum number of photons has been detected within a specific time span to result in a voxel detection. The problem with this, however, is that it still takes a long time for the sensor to evaluate whether an event has occurred. Even the most recent versions, such as the Prophecy sensor, still take over 1 μs. Furthermore, in such an embodiment, it is nearly impossible to distinguish a "false positive" due to ambient light or thermal noise from a true event.

[0006] WO2013 / 018006 describes a detector array with SPAD detectors for detecting incident photons. However, this system is not suitable for achieving the objective of the present invention of reducing false positives. WO'006 uses a collection of signals from SPADs arranged in rows and columns to determine the exact position in this way without reading each SPAD individually. However, this still generates a lot of signal from environmental noise, etc., which cannot be filtered out.

[0007] US 2012 / 257789 describes signal processing of events derived from sensor outputs to recognize motion and identify clusters of motion according to spatial / temporal criteria. However, the inputs used for this are only processed in post-processing of the signals, which is avoided in the present invention (by pre-processing). Therefore, US '789 "blindly" passes all sensor data through, so the bottleneck is still in post-processing.

[0008] US Patent Application Publication No. 2018 / 262705 describes a technique for detecting defective pixels that can use time-based conditions, but this application is not entirely suitable for solving the problem of the present invention.

[0009] Neither document describes SPDs or SPADs at all. On the other hand, the problem solved by the present invention is mainly related to the inherent function of SPDs, and in particular to SPADs, namely, the large amount of noise caused by ambient light and the like.

[0010] Currently, there are no systems with sensor architectures that can achieve the desired detection rate under the proposed capabilities, such as the photon budget.

[0011] The present invention aims to find a solution to at least some of the above problems. Summary of the Invention

[0012] The present invention relates to a high-speed sensor system as claimed in the patent. The architecture is a system that uses highly sensitive sensors, such as single-photon avalanche diodes (SPADs), to detect events with a very low photon budget, allowing for very short scanning times to determine true detections versus false detections due to ambient light or thermal noise. The combination of these features allows for high-speed imaging even at high resolution. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram of a basic version of the system according to the present invention; [Figure 2] 1 is a schematic diagram of a system (1) according to an embodiment of the present invention. [Figure 3] 1 is a schematic diagram of a photosensitive zone equipped with detectors and connected to row and column buses. [Figure 4A] 4 is a diagram illustrating a further embodiment based on FIG. 3. FIG. [Figure 4B] 4 is a diagram illustrating a further embodiment based on FIG. 3. FIG. [Figure 4C] 4 is a diagram illustrating a further embodiment based on FIG. 3. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Unless otherwise defined, all terms used in the description of the present invention, including technical and scientific terms, have the meanings commonly understood by those skilled in the art to which the present invention pertains. In order to better understand the description of the present invention, the following terms are clearly explained.

[0015] As used herein, "a" and "the" refer to the singular and plural unless the context requires otherwise. For example, "a segment" means one or more segments.

[0016] When the term "around" or "about" is used herein in conjunction with a measurable quantity, parameter, period of time, or point in time, it means that, to the extent that a variation in the numerical value is applicable to the described invention, such variation is no more than about 20%, preferably no more than 10%, more preferably no more than about 5%, even more preferably no more than 1%, and even more preferably no more than about 0.1% of the cited numerical value, provided that when the term "around" or "about" is used, it should be understood that the numerical value of the quantity used is itself specifically disclosed.

[0017] The terms "comprise," "comprising," "consist of," "consisting of," "provided with," "have," "having," "include," "including," "contain," and "containing" are synonymous and are inclusive or open terms that indicate the presence of what follows the term and do not exclude or preclude the presence of other components, features, elements, members, or steps that are known from or disclosed in the prior art.

[0018] When a numerical interval is recited using endpoints, the recitation includes all integer, fractional, and / or real numbers between and including those endpoints.

[0019] In a first aspect, the present invention relates to a high-speed image sensor system, the system comprising one or more light sources and an array having a plurality of single photon detectors (SPDs), the detectors preferably being spatially distributed across the array in a substantially matrix format, at regular intervals from one another, the SPDs being capable of detecting single photons and registering the detection of a single photon by a detection signal; the system further comprising a plurality of row buses and column buses, the detectors being grouped into rows and / or columns, the detectors per row being connected to one or more row buses and / or the detectors per column being connected to a column bus that aggregates signals from the detectors. the row buses and the column buses are connected to one or more column buses on the optical fiber, and for each row bus, only the detectors in one of the rows are connected to the row bus that aggregates signals from the detectors, and for each column bus, only the detectors in one of the columns are connected; the system further includes an evaluation circuit, the row buses and the column buses are connected to the evaluation circuit, the evaluation circuit being configured to evaluate the aggregated signals from the row buses and the column buses against a predetermined confirmation pattern to confirm detection and location of an incident photon, the confirmation pattern including temporal and / or spatial conditions, and detection being confirmed based on the signals on the row buses and the column buses following the predetermined pattern.

[0020] Until now, single-photon detectors have been rarely used due to their high sensitivity, which means that false positives have an excessive impact on imaging. Furthermore, removing these false positives requires further processing of the SPD output, thereby reducing processing speed and effectively negating the benefits of SPDs. The present invention successfully limits the input data to be processed by grouping sensors and aggregating their individual signals (from row and column buses) before processing them, while filtering out false events based on predefined confirmation patterns to filter out false positive detection signals. In this way, the (relative) location of the detecting sensor can also be determined from the aggregated information. An evaluation circuit then checks the signals from the (row and column) buses and evaluates whether they match the confirmation patterns to filter out false positives. These confirmation patterns may include spatial and / or temporal conditions, with the spatial conditions being related to the (relative) location of the sensors (and / or their column and / or row buses) whose signals indicate a "detection." For example, if sensors (such as those in columns 17 and 18) receive a "detection" signal from two adjacent column buses (and / or two row buses), the signal will be considered as a "detection." When a signal indicating "detection" is received, it can be verified that this is not a random false detection due to, for example, thermal noise. Combined or not, further temporal conditions can be imposed relative to the time (absolute and / or relative) at which the signal indicates "detection", such as a requirement that a detection be confirmed by interpreting this as a true event and "detection" only if two (or more) signals indicating "detection" are received within a given time period.

[0021] Applicant notes that in most prior art systems, either the evaluation circuitry is unable to accept and process large amounts of input data, and is no longer able to perform within the planned 10 ns timeframe, or the sensor itself is modified to pre-process the data, requiring more sophisticated sensors or add-ons to enable this functionality, resulting in significantly less practical and / or more expensive solutions. Furthermore, it is not possible for the sensor itself to reliably process signals without considering inputs from other sensors. The solution to this is that all sensors must be interconnected with respect to their inputs, which, as mentioned above, has significant impacts on cost as well as energy efficiency, speed, and compactness.

[0022] In a preferred embodiment, the spatial condition includes detecting an aggregate signal including detection signals from at least two column buses associated with adjacent columns in a predetermined time window, and detecting an aggregate signal including detection signals from at least two row buses associated with adjacent rows in a predetermined time window.

[0023] Spatial conditions can be applied, for example, in the form of so-called "coincidence detection," which compares the (relative and / or absolute) physical locations of the sources of multiple "positive" signals to confirm them based on probabilistic estimation. For example, in some situations, neighboring or adjacent column or row groups can be expected to have a (substantially) simultaneous (same time span) chance of detection signals (by imposing additional restrictions on the "length," i.e., the number of sensors per group), which can be negligible under, for example, limited ambient light conditions. In other situations, the conditions to avoid false detections can be more stringent, for example, requiring three or more adjacent groups to show confirmed detection. Additionally or alternatively, requirements can be set regarding the bus signal strength (threshold) necessary to constitute a true detection, which further depends on the sensor type and sensitivity as well as sensor gain variations.

[0024] Preferably, the relative positions of detectors relative to the column and row buses are utilized to enable verification of spatial conditions. Typically, by regularly subdividing the array with a constant number of detectors per column bus and a constant number of detectors per row bus (regardless of whether the number of row and column buses is equal), logic programming for verifying spatial conditions can be greatly simplified. Once a detector signal is identified, its valid position can be easily determined from the known positions of the detectors on the row and column buses. Aggregating signals from multiple detectors per column or row bus into a single signal greatly reduces computational requirements. A drawback of this is that some information is usually lost; detections are only identified for a group of detectors connected to a column or row bus, and it is not possible to determine which detector or detectors are specifically involved. However, by doing this on both the row and column buses, detections can be located along the column bus (corresponding to the X coordinate of the array) and along the row bus (corresponding to the Y coordinate of the array), providing an approximate two-dimensional position for the detector location on both buses.

[0025] In a preferred embodiment, the temporal condition is detecting an aggregate signal that includes detected signals from at least one row bus, preferably from at least two row buses associated with adjacent rows. and a temporal overlap of detecting an aggregate signal including detection signals from at least one, preferably at least two, column buses associated with adjacent columns, wherein the position of incidence of the photon on the array is determined based on the column bus and row bus associated with the confirmed detection.

[0026] Additionally, the time conditions can be set to be stricter or looser depending on the situation, and can even be set dynamically so that in certain situations they are automatically adjusted based on, for example, the average number of bus signal detections per time unit (regardless of whether they are false positives or not).

[0027] Generally, the time requirement requires that separate detection signals from the row and column buses overlap to confirm detection, meaning that the leading edges of the detection signals fall within a predetermined time span of each other, e.g., less than 100 ns, less than 50 ns, preferably less than 10 ns, 5 ns, 2.5 ns, 2 ns, 1.5 ns, 1.0 ns, or even less. Preferably, the detection signals overlap to a significant extent (at least 25%, preferably at least 50%, or even 75%). This requirement ensures that it is statistically unlikely that a random signal, such as thermal noise, would be detected by two different detectors at approximately the same time. To increase this reliability, the predetermined time span and / or the degree of overlap and / or other factors can be narrowed. Based on this, if detections occur on multiple row and / or column buses, it can also be verified whether separate events are involved. Finally, when detections occur on multiple row buses and multiple column buses, accurate matching can be achieved between different column and row buses by comparing the degree of overlap. For example, if a signal is detected on column bus 208 at t=0.075 ns and on column bus 472 at t=2.45 ns; and if a signal is detected on row bus 171 at t=2.38 ns and on row bus 23 at t=0.81 ns, two events can be detected with near-correct accuracy: one at location (row bus / column bus) 208 / 23 and one at location 472 / 171.

[0028] A second temporal correlation is applied between successive time windows. Consequently, using a scanning system, if a correct detection occurs in time window 1, say at detection location [10,100], then the predicted detection coordinates in time window 2 are expected to be close to this detection location. In this way, detections that are far from the previously detected coordinates can be taken into account and considered to confirm their authenticity with subsequent information. In this way, spatiotemporal patterns can be searched for in the detections across multiple time windows that match or closely approximate expected patterns following the light source's scanning pattern.

[0029] The concept of the present invention is to reduce the computational load in this way, aggregate the detector signals to enable the rapid detection of valid events for a given bus, and then exploit the newly imposed constraints to search for the exact location of the event.

[0030] In a particularly preferred embodiment, the evaluation circuitry comprises one or more simple circuits that evaluate the aggregate signals from two or more adjacent row buses / column buses and confirms detection only if the aggregate signals from two or more adjacent row buses indicate "detection" and / or the aggregate signals from two or more adjacent column buses indicate "detection." For example, the evaluation circuitry may be provided as separate circuits that take as input the aggregate signals from two (or more, if necessary) row buses / column buses and confirm them if two (or all N) are positive (i.e., indicating detection). One way to do this is with an AND logic gate, i.e., a switch circuit. The use of multiple transistors as detectors, which will only produce a detector output if there are two detector inputs. Each "group" of buses (based on spatial criteria) is connected to a transistor that can confirm or reject detection for that group. For example, if two or more adjacent buses display a "positive" aggregate signal for confirmation, this can be done with multiple transistors connected in series. Other ways to achieve this also form part of the invention, and the above example is for illustrative purposes only.

[0031] Detections are confirmed by evaluation circuits that can easily determine the physical location on the array (row and column buses allow simple XY coordinate positioning), record true detections and pass them on to the processing layer, while not passing on false detections.

[0032] This allows for the definition of a macropixel, consisting of a SPAD of size MxN, within which a confirmed detection can be located. For example, if aggregate signals indicating a detection occur at the same time in rows 17 and 18, and also at the same time in columns 33 and 34, then the detection is confirmed in macropixel (17-18; 33-34) of a 2x2 SPAD array. Based on spatial requirements, the size of the macropixel can vary: 2x1, 1x2, 3x2, 2x3, 3x3, 3x1, 1x3, etc. Furthermore, in some cases, the location of the confirmed detection can be narrowed (in post-processing) to the center of the macropixel, especially when more than two adjacent detections are required.

[0033] In certain embodiments, the evaluation circuitry adjusts between various spatial and / or temporal conditions for confirmation, i.e., adjusting the spacing between adjacent aggregate signals to indicate a detection (by adding circuitry that converts the "positive" aggregate signal or the "positive" signal per detector into a controllable signal with a fixed time length and amplitude), or adjusting the number of adjacent aggregate signals to indicate a detection.

[0034] WO'006 describes a SPAD array in which each row and column of SPADs is connected to an OR gate, and the row and column whose signal is detected are combined to identify the photon impact. Of course, this does not take into account the current problem, i.e., false detections, which transmit a lot of noise to the processing layer. The present invention eliminates these false results by imposing an additional condition, i.e., the condition of multiple detections by two or more adjacent rows / columns.

[0035] Particularly preferably, the spatial condition comprises detecting an aggregate signal comprising detection signals from at least two column buses associated with adjacent columns within a predetermined time window, and detecting an aggregate signal comprising detection signals on at least two row buses associated with adjacent rows within a predetermined time window, and the confirmation pattern comprises temporal and spatial conditions.

[0036] Imposing both spatial and temporal constraints greatly improves the reliability of detection confirmation, and also allows for accurate interpretation of multiple events as described above.

[0037] In a preferred embodiment, each SPD is uniquely connected to one column bus and one row bus. The advantage of this is that the number of detectors is limited and their location is more precisely determined (in X and Y coordinates). For example, in a row of 20 SPDs, the first five SPDs are connected to the first row bus for that row, the second five to the second row bus for that row, and each of the next set of five is connected to the third and fourth row bus for that row. Thus, each SPD is only connected to one row bus, but conversely, the row bus is connected to multiple SPDs.

[0038] In a preferred embodiment, each SPD is unidirectionally connected to either one column bus or one row bus. Such a system would have a simpler architecture but would use more detectors. However, a major advantage is that false detections of such detectors are typically not propagated to both the column bus and row bus associated with the detector, as in the previous embodiment, even though "false" detections (e.g., due to ambient light) can be eliminated through other conditions (such as the requirement that two or more adjacent buses receive the detection signal).

[0039] In a further preferred embodiment, the array includes M×N single photon detectors (SPDs), the SPDs being arranged in a matrix of M rows and N columns, the SPDs being distributed row by row, preferably consecutively within that row, across two or more row groups of SPDs, the same distribution of SPDs per row being consecutive across all rows, the SPDs in a row group being connected to a row bus associated with that row group, and the spatial condition further including detecting an aggregate signal including detection signals of at least two row buses associated with adjacent row groups within a predetermined time window. For example, there may be up to M row buses and N column buses, although in practice there may be far fewer since multiple SPDs are grouped into row and column buses.

[0040] In certain situations where the number of detectors per row can become very large, and the number of detectors per row can be very large (preferably, each detector can belong to only one row group, although some or all detectors can optionally belong to more than one row group), further dividing each row into multiple row groups, each with its own row bus, can provide more efficient computations. Preferably, each bus manages 1 to 20 detectors, more preferably 2 to 10, even more preferably 2 to 5, and even more preferably 2 to 4 detectors. This limits the number of multiple events appearing on the row bus per time span, thereby requiring less computing power for this differentiation. Furthermore, with a very large number of detectors per row that are not subdivided into row groups, there is an increased chance that multiple detectors will detect events quasi-simultaneously and subsequently fail to differentiate, or reliably differentiate, when the signals are aggregated onto the row bus. Proper selection of the number of detectors per row group can significantly reduce this problem. The optimal number can be determined based on statistical observations, but also depends on the application of the array (high or low ambient light). In particular embodiments, the number of detectors per row group can be adjusted dynamically or manually to adapt to the situation, and therefore there may be many predefined distributions for which ranges to choose.

[0041] In a further preferred embodiment, the array includes M×N single photon detectors (SPDs), the SPDs are arranged in a matrix of M rows and N columns, the SPDs are distributed across column groups of two or more SPDs, column by column, preferably consecutively in that column, the same distribution of SPDs per column being consecutive across all columns, the SPDs in a column group being connected to a column bus associated with that column group, and the spatial condition further includes detecting an aggregate signal including detection signals of at least two column buses associated with adjacent column groups within a predetermined time window.

[0042] The reasons given earlier for dividing the row detectors into row groups also apply in this case. It is particularly preferred that both distributions (row groups and column groups) are combined as described above.

[0043] Alternatively, detectors may be assigned to multiple row and / or column buses in an alternating manner, which can provide advantages for coincidence detection. For example, a hit that triggers multiple adjacent detectors in a row (which occurs frequently) will result in a "detection" on more than one row bus associated with that row. In a continuous distribution, this will only trigger one row bus (unless it involves exactly two detectors located at the transition between two row groups).

[0044] In a preferred embodiment, the SPD is configured to provide a signal to an associated row bus and / or column bus, said signal being binary in nature and indicating whether the SPD has detected an incident photon.

[0045] In a preferred embodiment, the system is configured to precharge the row and column buses at regular intervals, and the row and column buses are configured to discharge upon receiving an aggregate signal that includes the detection signal.

[0046] In a preferred embodiment, the evaluation circuitry is configured to check spatial consistency, whereby the last determination of the position at which a photon is incident on the array is compared with one or more previous position determinations, and if the evaluation circuitry detects a spatial discrepancy between the last position determination and the previous position determination, whether dynamic or non-dynamic, that exceeds a predetermined upper limit, the evaluation circuitry rejects the last position determination.

[0047] The above adjustments based on historical data, i.e., the "known" incident positions of the reflected beams, allow for rough filtering to be performed to remove spurious signals, without consuming significant computing power in subsequent processing.

[0048] In a preferred embodiment, the system includes a synchronization component that synchronizes the signals from the detectors.

[0049] To determine mismatches between the detector and the light source, a synchronization component is provided to align and synchronize them in time. This can be achieved, among other things, by physically connecting the elements (detector and light source) via a cable and providing a synchronization signal to the elements via the cable. Alternatively, the optical signal (from the light source) itself can provide a synchronization moment that can be obtained during acquisition. In particular, this can be done by pulsing the optical signal according to a predetermined pattern that is "recognized" as the start of a scan. All sensors can use this as local time 0, thereby synchronizing them with each other and with the light source.

[0050] In a preferred embodiment, the system is configured to consider only signals from a variable subset of the detectors of the evaluation circuit when imaging during a portion of the imaging procedure, and not consider signals from detectors of the subset, the system selecting the detectors of the subset based on a previous position determination and optionally based on the orientation and / or position of one or more light sources, the detectors of the subset comprising up to 25%, preferably up to 10%, of the total number of detectors, and the detectors of the subset being grouped together.

[0051] In reality, most of the detectors on the array will not experience an event during each time span. Configuring the system to process signals from only some of the detectors in the evaluation circuit (i.e., effectively ignoring some) can significantly reduce the computational load. The problem lies in the fact that it is not possible to determine with certainty in advance which detectors to ignore. Therefore, based on statistical methods and historical data, and preferably also on the scanning pattern of the light source, it is possible to estimate the possible variations in the incident position of one or more beams reflected on the array relative to previously detected incident positions. Taking this into account, it is possible to define a region of interest where the reflected beam will strike with a given statistical probability, without considering detectors outside this region of interest. This region of interest may be substantially circular, elliptical, rectangular, or have a jagged pattern. In this way, particularly for very large arrays, a significant portion of the detectors need not be considered during processing. The determination of the region of interest may, in certain embodiments, take into account detections in several preceding time spans. This allows for the recognition of patterns that can be taken into account when predicting the region of interest. do.

[0052] A huge additional benefit of limiting to the subset of detectors that are active / considered is that this also leaves out the majority of false detections (due to thermal noise etc.).

[0053] In a preferred embodiment, the system is configured to activate only a variable subset of the detectors when imaging during a portion of the imaging procedure and not deactivate any of the detectors of the subset, the system selecting the detectors of the subset based on a previous position determination and optionally based on the orientation and / or position of one or more light sources, the detectors of the subset comprising up to 25%, preferably up to 10%, of the total number of detectors, and the detectors of the subset being grouped together.

[0054] In a preferred embodiment, the detector comprises a single photon avalanche diode (SPAD), preferably a SPAD.

[0055] SPADs are semiconductor photodetectors that use a very high reverse voltage to cause impact ionization during detection, leading to an avalanche effect and a rapid buildup of high current. SPADs are chosen in part based on the speed at which impact ionization is achieved and the ease with which these detectors can be "reset."

[0056] In a preferred embodiment, the detector includes a quenching circuit that normalizes the detection signal from the detector.

[0057] A quenching circuit is required to reduce the signal (avalanche current), reset the detector and limit the signal caused by the avalanche current. The quenching circuit is preferably active, for example using a so-called discriminator to reduce the reverse voltage, but may alternatively be passive in the form of a resistor in series with the detector.

[0058] In a preferred embodiment, up to 100 detectors, preferably up to 50 detectors, are connected per row bus and per column bus.

[0059] As mentioned above, it is more efficient to limit the number of detectors per bus, as this makes it easier to distinguish multiple events on a bus from one another and allows for stricter spatial constraints to be imposed.

[0060] Based on a confirmed detection, a timestamp for the incident photon is determined at the processing layer only for that signal, whereas in prior art systems this occurs for all signals, including false detections. The determination of the timestamp can be done in many ways, depending on the difference in timestamps of the separate aggregate signals that together led to the confirmed detection. For example, the earliest signal can always be selected to signify that there was already an impact at that point. Alternatively, the average of the timestamps associated with the confirmed detections may be selected instead of the middle or last timestamp to resolve (or at least mitigate) hardware errors. Finally, additional information is obtained from the time differences between timestamps associated with different rows and columns, which provides further insight into the incident photon.

[0061] In the following, some specific embodiments will be described by way of example and with reference to schematic diagrams. [Example]

[0062] In a first embodiment, the present invention provides very fast (i.e., sub-10 ns time span) and and detection architectures adapted to low-threshold (i.e., limited photon budget) photon detection, and their conversion into unambiguous digital signals representing events (i.e., effective detection of the reflected light beam rather than ambient light or thermal noise).

[0063] 1 shows a schematic diagram of a basic version of the system according to the invention, which includes at least one light source (2) and one, preferably N, detectors, or sensors (1), that capture a particular scene in an environment. The light source typically illuminates the scene with a light beam (laser or LED) according to a pre-programmed pattern and / or in a sequential manner, such as by scanning the scene through light spots (3) in a dynamic pattern (4). The N detectors (1) detect the position of the light spots (3) within the scene, for example by triangulation based on the outputs of the different detectors.

[0064] Since the invention aims to process millions of voxels per second, we need to avoid tackling the problem blindly, and the scanning speed needs to be very fast, optimizing the process given a limited photon budget and short timeframe.

[0065] 2 shows a schematic diagram of a system (1) according to an embodiment of the present invention, which includes a photosensitive zone (11) typically comprising an array of photosensitive elements or detectors (e.g., SPADs). The system further includes an evaluation circuit (12) including logic circuits, such as logic gates or variants thereof, connected to the photosensitive zone (11) and configured to determine the validity of data obtained from the detectors in the photosensitive zone. In this case, the system further includes a logic circuit (12) for fusing the data / signals obtained from the detectors into a (single) data stream (when combined with a logic circuit for validating the data). The system may further include a synchronization component for synchronizing the data stream with the optical signals from the light source and / or the data from the different detectors.

[0066] A first, more specific version of this embodiment is conceptually shown in FIG. 3, in which the photosensitive zone (11) includes a plurality of SPADs (21) arranged in an array of M rows and N columns, each SPAD including a passive or active quenching circuit, represented as a square cell. Two connection points per cell are clearly visible and serve as connection points from the SPAD to row and / or column buses. All SPADs are connected to a column bus (24) and a row bus (23), with one or more buses provided per row and / or column (it is possible to connect all SPADs in a row or column to one bus, but multiple buses per row and column are preferred). In FIG. 3, two row buses are used per row and two column buses are used per column, with SPADs alternately connected to one or the other bus. Alternatively, this could be done sequentially (the first X consecutive SPADs to the first bus, the remaining to the second bus).

[0067] When a photon is detected on a SPAD, this leads to the detection of an event occurring on the associated row and column buses.

[0068] Figure 4A shows an example of a photon impact detected on four different adjacent SPADs (see circles). If there is an impact sufficient to trigger detection in the SPAD (lower energy impacts may not be sufficient to trigger avalanche breakdown in the SPAD), the detection is propagated to four row (group) buses and four column (group) buses, as indicated by the signals on the left for the row buses and the signals at the bottom for the column buses.

[0069] In Figure 4B, a single SPAD (top left, second row, first column) also detects an event and communicates the detection to one row bus and one column bus.

[0070] The evaluation circuit (12) receives signals from the row and column buses and imposes several conditions for evaluating the incoming signals to indicate whether they are confirmed events. For example, the evaluation circuit will typically consider a single event, such as a top left detection, to be a false positive, while multiple detections on adjacent buses (e.g., rows 3-4, columns 3-4) will typically be considered a true event.

[0071] In Figure 4C, an additional single event is detected in row 5, column 2. This leads to the inference of events on two adjacent column buses (column buses 2 and 3), which stem from two separate events. By considering the row buses, we also see that these are separate single events, and we can consider both single events.

[0072] To avoid a single event being able to satisfy the evaluation circuit's criteria (predetermined pattern or patterns), it is advantageous to have enough buses per row and column of the SPAD, thereby limiting the chance of accidentally observing an event on an adjacent bus in the same time frame.

[0073] However, a true event (i.e., a true impact of the light beam reflected on the array) will usually be a detection that satisfies the criteria of the evaluation circuitry, and the resulting trigger will not be limited to one SPAD but will trigger one or more adjacent SPADs, and often in a very short time frame, so that correlations between individual detections are identified when evaluated.

[0074] Thus, in accordance with the present invention, coincidence detection based on the spatial location of the pixels (detectors) of the array is utilized by looking for confirming patterns in the signals on the row and column buses to determine whether a true photon packet (and therefore a reflected light beam) has been detected.

[0075] In Figure 4C, an additional measure is provided: a so-called "enable window" (22) limits the active and considered detectors to a subset of detectors. The term "enable window" is not limited in shape or size here, but can vary dynamically. This window serves as a prediction zone in which the detection of a light point is predicted based on historical data (previous detections, movement patterns between previous detections, ambient light intensity, etc.). In this way, a large portion of the SPADs is not taken into account for determining the incident position of the light beam, which, on the one hand, significantly reduces false detections (since they are almost randomly distributed and limiting the number of SPADs also limits the number of false detections to the same extent), and, on the other hand, limits the number of signals processed, thereby avoiding bus saturation by a single event.

[0076] To avoid saturation of the column and / or row buses in this way, alternatively or additionally, the sensitivity of the detector can be dynamically or non-dynamically adjusted. As a result, more energy must be dissipated in the active optical signal (i.e., the light source), increasing the probability that enough photons collide together (simultaneously), thereby effectively filtering background radiation and thermal noise.

[0077] By defining several confirmation patterns, the system can easily and quickly (and dynamically) adapt to different situations. For example, spatial conditions can be eliminated or relaxed in situations where ambient light is limited, such as at night or indoors. "Unwanted" events can also be filtered out by, for example, taking into account the pattern in which the light source scans the scene and comparing the "detection" immediately after it occurs with the expected lighting pattern. Thus, in one embodiment, the light source may be programmed to scan the scene according to a Lissajous pattern, which impacts the observed detections and typically adopts a similar pattern, albeit with distortions. In this case, detections that deviate significantly from the expected lighting pattern can be rejected as noise, etc.

[0078] Additionally, in certain embodiments, multiple "confirmation flows" of evaluated detections that match different confirmation patterns may be kept in parallel so that these can be cross-checked with measurements at later timespans to obtain greater accuracy as to which is the correct confirmation flow.

[0079] Communication from the detector to the bus is, for example, via pull-down (or pull-up) elements on the bus, whereby detection generates a signal that ensures the discharge of the precharged bus. At the end of each detection time span, the bus is precharged again, waiting for a detection by the SPAD to occur. The advantage of this is that the discharge of these pull-downs can be detected very easily by the evaluation circuit. If necessary, an amplification circuit can be provided before discretizing the signal.

[0080] Based on this, the system can provide a list, by time span, of the locations in the array where the identifying pattern was detected.

[0081] In another embodiment, the invention relates to a system in which the photosensitive portion is implemented in a first layer and the processing and logic circuitry is implemented in a second layer. The layers are stacked on top of each other and connected by one or more electrical connections for each pixel or group of pixels. The search for the expected detection pattern of the sensor can now be performed more locally, so that instead of the detection pattern projected (on the row / column bus), a two-dimensional pattern can be searched in each subwindow of the sensor array.

[0082] The present invention further relates to a high-speed image sensor system, the system comprising one or more light sources and an array having a plurality of single photon detectors (SPDs), the detectors being spatially distributed throughout the array in a substantial matrix format, preferably at regular intervals from one another, the SPDs being capable of detecting single photons and registering the detection of a single photon by means of a detection signal; the system further comprising a plurality of buses, the detectors being grouped into clusters of two or more detectors, the detectors of each cluster being connected to one bus that aggregates the signals from the detectors of the cluster, with only the detectors of one of the clusters being connected to each bus; the system further comprising an evaluation circuit, the evaluation circuit being provided with the relative positions of the clusters and the buses being connected to the evaluation circuit, the evaluation circuit being configured to evaluate the aggregated signal from the bus against a predetermined confirmation pattern to confirm the detection and position of an incident photon, the confirmation pattern comprising temporal and / or spatial conditions, the confirmation pattern being based in particular on the relative positions of the clusters, and detection being confirmed by the signal from the bus following the predetermined pattern.

[0083] Another embodiment of the present invention introduces a stack layer where the trigger signals of the SPAD detectors are combined and analyzed. For single-layer operation, assertions need to be monitored either in "projections" on the column and row buses, or in different projections such as diagonal or other buses. The introduction of a second layer provides more localized monitoring of clusters of SPAD detectors, allowing for simultaneous event generation monitoring. This more localized monitoring allows fewer SPAD detector devices per cluster to be considered, avoiding bus saturation when a large number of environmental events are occurring.

[0084] Furthermore, the assertion pattern can be considered as a two-dimensional assertion pattern that can be compared with each detection cluster under test. If a match with the required assertion pattern is found, the detection can be considered with high confidence.

[0085] In another embodiment, the invention relates to an array of pixels, each pixel containing one or more single photon detectors, preferably SPADs. The pixels may (partially) overlap, allowing adjacent pixels to share a detector.

[0086] Thermal noise and background radiation incident on a pixel causes the detector to emit a detection signal at a rate equal to the DCR (dark count rate) and BGR (background rate). These events are unrelated in time, so the detectors are triggered separately. Because false positive events are uncorrelated, the incorporation of false positives is significantly reduced by including in the confirmation pattern, i.e., temporal and spatial conditions, that a detection is confirmed only if one or more SPADs in a pixel are triggered in a given time frame.

[0087] In contrast, if a pixel is illuminated with an active light beam (i.e., resulting from illumination of the scene by a light source) within a given time span (e.g., 10 ns), and the energy budget of the light beam is adjusted so that the SPAD or other detector illuminated by the reflected beam effectively detects and triggers one or more photons, the probability that the pixel will detect multiple events on its separate detectors, i.e., SPADs, also increases significantly. This increases the reliance on coincidence detection. In the present invention, the coincidence detection principle is used as the signal does not provide an accurate timestamp associated with the event, since only the location of the event on the array is required.

[0088] The pixel can thus detect coincidence events and communicate them to the column and row buses to indicate that an event has occurred.

[0089] There can be a coincidence evaluator for each pixel or set of pixels. Alternatively, the coincidence evaluators can be integrated into the periphery of the system, which greatly simplifies the array itself, since there is no longer a need for local coincidence evaluators, except for the quenching circuitry. Each SPAD communicates its observations to the row and / or column bus with which it is associated. The periphery can now contain massively paralleled digital circuits that monitor the state of the bus every predetermined time span (10 ns). During this time span, events such as: - two adjacent columns and identical lines (when a 2x1 checking pattern is sufficient); - two adjacent lines and identical columns (for a 1x2 checking pattern); - two adjacent lines and two adjacent columns (for a 2x2 checking pattern); or - Different predetermined confirmation patterns; When detected in, the event is acknowledged and reported, then optionally time-stamped and propagated to subsequent devices.

[0090] The present invention should not be construed as being limited to the embodiments set forth above, and certain modifications or variations can be made to the described examples without the need to reevaluate the scope of the appended claims.

Claims

1. 1. A high-speed image sensor system, the system including one or more light sources and an array having a plurality of single photon detectors (SPDs), the detectors preferably being spatially distributed across the array in a substantially matrix format, at regular intervals from one another, the SPDs being capable of detecting single photons and registering the detection of a single photon by a detection signal; the system further including a plurality of row buses and column buses, the detectors being grouped into rows and / or columns, the detectors per row being connected to one or more row buses and / or the detectors per column being connected to one or more column buses that aggregate signals from the detectors. or a plurality of column buses, with each row bus having only one detector in each row connected to the row bus that aggregates signals from the detectors, and with each column bus having only one detector in each column connected; the system further comprising an evaluation circuit, the row bus and the column bus connected to the evaluation circuit, the evaluation circuit configured to evaluate the aggregated signals from the row bus and the column bus against a predetermined validation pattern to validate detection and location of an incident photon, the validation pattern including temporal and / or spatial conditions, and detection being validated based on the signals on the row bus and the column bus following the predetermined pattern; a high-speed image sensor system, wherein the spatial condition includes detecting an aggregate signal including detection signals from at least two column buses associated with adjacent columns in a predetermined time window and / or detecting an aggregate signal including detection signals from at least two row buses associated with adjacent rows in a predetermined time window, and the temporal condition relates to a temporal overlap between detecting an aggregate signal including detection signals from at least one row bus and detecting an aggregate signal including detection signals from at least one column bus, and wherein a position of incidence of a photon on the array is determined based on the column bus and row bus associated with a confirmed detection.

2. 2. The high-speed image sensor system of claim 1, wherein the spatial conditions include detecting an aggregate signal including detection signals from at least two column buses associated with adjacent columns in a predetermined time window, and detecting an aggregate signal including detection signals from at least two row buses associated with adjacent rows in a predetermined time window.

3. 3. The high-speed image sensor system of claim 1, wherein the evaluation circuitry, provided as parallel-connected digital circuits, is peripherally coupled to the row and column buses, and the evaluation circuitry monitors the state of the row and column buses to find a detection signal in an aggregate signal per predetermined time span, and upon detecting a detection signal in two adjacent column buses and / or two adjacent row buses, the detection is confirmed and reported.

4. 4. The high-speed image sensor system of claim 1, wherein the temporal condition includes a temporal overlap between detecting an aggregate signal including detection signals from at least two row buses associated with adjacent rows and detecting an aggregate signal including detection signals from at least two column buses associated with adjacent columns, and the incident position of a photon on the array is determined based on the column bus and row bus associated with the confirmed detection.

5. 5. The high-speed image sensor system of claim 1, wherein each SPD is uniquely connected to one column bus and to one row bus.

6. The array comprises M×N single photon detectors, the single photon detectors being arranged in a matrix of M rows and N columns, the SPDs for each row being distributed across a row group of two or more SPDs, preferably consecutively in the row, the same distribution of SPDs per row being consecutive across all rows, the SPDs of a row group being connected to a row bus associated with the row group, and the spatial condition providing an aggregate signal comprising detection signals of at least two row buses associated with adjacent row groups.

6. The high-speed image sensor system of claim 5, further comprising detecting in a predetermined time window.

7. 7. The high-speed image sensor system of claim 5, wherein the array includes M×N single-photon detectors arranged in a matrix of M rows and N columns, the SPDs for each column are distributed across two or more column groups of SPDs, preferably consecutively in the column, the same distribution of SPDs for each column being consecutive across all columns, the SPDs of a column group being connected to a column bus associated with the column group, and the spatial condition further includes detecting an aggregate signal including detection signals of at least two column buses associated with adjacent column groups in a predetermined time window.

8. 8. The high-speed image sensor system of claim 1, wherein the SPD is configured to provide a signal to the connected row bus and / or column bus, the signal being binary in nature and indicating whether the SPD has detected an incident photon.

9. 9. The high-speed image sensor system of claim 1, wherein the system is configured to precharge the row buses and column buses at regular intervals, and the row buses and column buses are configured to discharge upon receiving an aggregate signal including a detection signal.

10. 10. The high-speed image sensor system of claim 1, wherein the evaluation circuit is configured to check spatial consistency, and the last determination of the position at which a photon is incident on the array is compared with one or more previous position determinations, and the evaluation circuit rejects the last position determination if it detects a spatial discrepancy between the last position determination and the previous position determination that exceeds a predetermined upper limit, whether dynamic or non-dynamic.

11. 11. The high-speed image sensor system of claim 1, wherein the system includes a synchronization component for synchronizing signals from the detectors.

12. 12. A high-speed image sensor system as described in any one of claims 1 to 11, wherein the system is configured to consider only signals from a variable subset of the detectors in the evaluation circuit when imaging during a part of the imaging procedure and not consider signals from detectors of the subset, the system selecting the detectors of the subset based on a previous position determination and optionally based on an orientation and / or position of one or more light sources, the detectors of the subset comprising at most 25%, preferably at most 10%, of the total number of detectors, and the detectors of the subset being grouped together.

13. 13. A high-speed image sensor system according to any preceding claim, wherein the system is configured to activate only a variable subset of the detectors when imaging during a portion of the imaging procedure and to not deactivate any of the detectors of the subset, the system selecting the detectors of the subset based on a previous position determination and optionally based on an orientation and / or position of one or more light sources, the detectors of the subset comprising up to 25%, preferably up to 10%, of the total number of detectors, and the detectors of the subset being grouped together.

14. 14. A high-speed image sensor system according to any preceding claim, wherein the detector comprises a single-photon avalanche diode (SPAD), preferably a SPAD.

15. 15. The high-speed image sensor system of claim 1, wherein the detector comprises a quenching circuit for normalizing a detection signal from the detector.

16. 16. A high speed image sensor system according to any preceding claim, wherein up to 100 detectors, preferably up to 50 detectors, are connected per row bus and per column bus.

17. 17. The high-speed image sensor system of claim 1, wherein the evaluation circuitry is configured to check spatial consistency, wherein a final determination of a position at which a photon is incident on the array is compared with one or more previous position determinations, and wherein the evaluation circuitry rejects the final position determination if it detects a spatial discrepancy between the final position determination and the previous position determinations, whether dynamic or non-dynamic, that exceeds a predetermined upper limit.

Images