Optical Sensing System

Abstract

The present invention relates to an optical sensing system (1) for three-dimensional imaging. The optical sensing system (1) comprises at least one optical sensor (2) including a plurality of sensing units (3). Each of the sensing units (3) comprises a photodetector. The optical sensor (2) further comprises a timing module including a plurality of timing means. Each of the sensing units (3) corresponds to one of the timing means, or the sensing units (3) are divided into groups, each group of the sensing units (3) corresponding to one of the timing means. The optical sensing system (1) further comprises at least one light source (5). The optical sensing system (1) further comprises an optical system (3) capable of generating an image of a scene (4) on the optical sensor (2). The optical sensing system (1) further comprises a reading module. Each timing means is adapted to measure at least one first depth value, said first depth value being mathematically related to the time between the emission of a light pulse by a light source (5) and the reception of said light pulse by an optical sensing unit (3). The reading module is adapted to read said first depth value from said timing means only if the corresponding sensing unit (3) or said group of sensing units (3) has a positive detection state.

Description

[Technical Field]

[0001] The present invention relates to optical sensing systems, and in particular to optical sensing systems suitable for operation at short and long ranges and suitable for suppressing occlusion. [Background technology]

[0002] Optical sensing systems are used in a variety of applications. Many of these systems have optimal performance at either short or long ranges. For example, some systems have optimal spatial resolution at short ranges, while others have optimal spatial resolution at long ranges.

[0003] Furthermore, optical systems that use triangulation have problems with occlusion due to the presence of objects, which creates blind spots for the sensor in such systems, where triangulation is not possible due to missing information.

[0004] Therefore, there is a need for an optical sensing system that can function at both long and short ranges and that can mitigate occlusion issues.

[0005] The present invention aims to solve some of the above problems. Summary of the Invention

[0006] An object of embodiments of the present invention is to provide an optical sensing system suitable for operation at both short and long distances and for reducing or preventing occlusion problems. A further object is to create a LIDAR imager capable of high spatial resolution without requiring a coaxial optical design, in which both the laser beam scanner and the detector are positioned consecutively along the same optical axis. A coaxial optical design dramatically complicates system design. An advantage of the present invention is that a coaxial design can be avoided by separating the scanning beam and replacing the single detector with a LIDAR imager capable of providing timestamp information for photons associated with the scanning beam. The above objects are achieved by the system and method according to the present invention.

[0007] In a first aspect, the present invention relates to an optical sensing system for three-dimensional imaging, the system comprising: - at least one optical sensor (2') including a timing module comprising a plurality of sensing units (3) and a plurality of timing means, each of the sensing units (3) including a photodetector, each of the sensing units (3) corresponding to one of the timing means, or the sensing units (3) being divided into groups, each group of the sensing units (3) corresponding to one of the timing means; - at least one light source; an optical system (14) capable of generating an image of the scene (8) on said optical sensor (2'); a reading module; It has. The timing module is adapted to measure at least one first depth value, the first depth value being mathematically related to the time between emission of a light pulse by the light source (5) and reception of the light pulse by the sensing unit (3). The reading module is adapted to read the first depth value from the timing means only if the sensing unit corresponding to the timing means or the group of sensing units corresponding to the timing means has a positive detection state.

[0008] Preferably, the system is adapted to filter false positives caused by thermal noise and ambient light from the outputs of the sensing units, and the reading module is adapted to read the first depth value from the timing means only if the filtered output of the sensing unit corresponding to the timing means, or the filtered output of the group of sensing units corresponding to the timing means, has a positive detection state.

[0009] Preferably, the filter exhibits either a spatial filtering method (i.e., at least two adjacent pixels have a positive detection simultaneously), a temporal filtering method (i.e., pixels having a repeated positive detection state when receiving, for example, repeated pulses over multiple time windows), or a combination thereof.

[0010] An advantage of embodiments of the present invention is that they provide reliable detection at long ranges based on time-of-flight measurements.

[0011] An advantage of embodiments of the present invention is that by reading out the first depth value from the timing means only when a sensing unit or group of said sensing units corresponding to the timing means has a positive detection, i.e. not reading all timing means in the system, power consumption of the timing means is minimal, which also results in fast acquisition of the first depth value.

[0012] An advantage of embodiments of the present invention is that the number of timing means is reduced when sensing units are grouped.An advantage of embodiments of the present invention is that the detection uniformity of the timing means is improved.

[0013] An advantage of embodiments of the present invention is that bulky optical lenses (used in, for example, LIDAR systems) are avoided.

[0014] Preferred embodiments of the first aspect of the present invention include one or suitable combinations of two or more of the following features:

[0015] The system is preferably adapted to calculate at least one second depth value, said second depth value being calculated based on the position of the light spot of the light source, preferably by triangulating data of the optical sensor with data of the light source, or by triangulating data of at least two optical sensors. The system preferably further comprises scanning means, said scanning means being adapted to scan the light beam from the light source over the scene along a trajectory. The system preferably comprises at least two optical sensors.

[0016] An advantage of embodiments of the present invention is that reliable detection is obtained even at short distances based on said triangulation.

[0017] An advantage of embodiments of the present invention is that depth measurements are obtained for all scenes, including blind spots, thereby reducing the problem of occlusion. Consider a system with two sensors according to the present invention organized along a baseline such that there is parallax between the two sensors. Consider a first laser beam scanner positioned near the first sensor so that the parallax between the light-emitting element (laser beam scanner) and the first sensor is minimized. In this case, an occlusion may occur that prevents the second sensor from seeing the laser beam. Because the first sensor and the light-emitting element have approximately the same viewpoint, the laser beam is not occluded and the first sensor does not experience such occlusion. In the event of occlusion, the second sensor does not see the laser beam, making it impossible to triangulate information from the first and second sensors to obtain a second depth measurement. However, because the first sensor is substantially near the light-emitting element and does not experience occlusion of the scanning laser beam, the first depth measurement, related to the time-of-flight, obtained by the first sensor is still available.

[0018] An advantage of embodiments of the present invention is that the first depth value and the second depth value should be similar or substantially similar or equal, thereby filtering out false detections and reducing or eliminating noise such as ambient light, resulting in more reliable detection.

[0019] Furthermore, if the sensors are organized as a stereo pair or system and there are at least two sensors, there will be at least two first depth values ​​from each sensor, and possibly at least two minus one triangulated second depth values ​​from each pairing of sensors in the system, or at least one triangulated second depth value based on all data simultaneously. For example, in the case of three sensors, such as #1, #2, and #3, triangulation can occur between #1 and #2, #1 and #3, and finally #2 and #3, and the best triangulation can be selected.

[0020] Each timing means is preferably substantially co-located with its corresponding sensing unit or units.An advantage of embodiments of the present invention is that the first and second depth values ​​are similar or equal, which can be imposed as a constraint in the filtering method.

[0021] The system preferably comprises at least two light sources, each of which corresponds to one of the optical sensors, and preferably each of which is co-located with its corresponding optical sensor. An advantage of embodiments of the present invention is that occlusion problems are reduced or eliminated.

[0022] The timing means preferably comprises a time-to-analog converter (TAC) adapted to output an analog value corresponding to the time between emission of said light pulse by the light source and reception of said pulse by the optical sensor. An advantage of embodiments of the present invention is that accurate time-of-flight measurements are obtained. When the timing means comprises a TAC, it is an advantage that the timing means can be implemented with significantly reduced circuit complexity.

[0023] Preferably, depending on the distance from the optical sensor (i.e. the distance between the optical sensor and the object from which the signal is reflected), the system is adapted to output the first depth value for longer detection distances from the optical sensor (e.g. equal to or greater than a predetermined distance) and to output the second depth value for shorter measurement distances from the optical sensor (e.g. less than a predetermined distance). Alternatively, a combination of both first and second measured depth values ​​may be used, e.g. each value may be weighted. An advantage of embodiments of the present invention is that reliable detection is obtained at both long and short distances.

[0024] The system is adapted to determine an initial estimate of a position of the light beam on the scene based on data from the sensing unit, and the system is adapted to determine a final estimate of a position of the light beam on the scene based on data from the light source. An advantage of embodiments of the present invention is that a low-resolution optical sensor can be used to calculate a rough estimate of the position of the light beam on the scene, where the accurate depth value is a first depth value measured using time-of-flight measurements. An advantage of embodiments of the present invention is that a compact and efficient system is obtained.

[0025] In a second aspect, the present invention relates to an optical sensing system for optical sensing, the system comprising: - at least one optical sensor including a timing module including a plurality of sensing units and a plurality of timing means, each of said sensing units including a photodetector; - at least one light source; - scanning means, preferably adapted to scan a light beam from said light source over the scene along a trajectory; an optical system capable of generating an image of the scene on the optical sensor; It has. The timing module is adapted to measure at least one first depth value, the first depth value being based on the time between emission of a light pulse by the light source and receipt of the pulse by the optical sensor. The optical sensor is adapted to calculate at least one second depth value, which is calculated by triangulation, preferably by triangulating data of the optical sensor with data of the light source, or by triangulating data of at least two optical sensors.

[0026] In a third aspect, the present invention relates to a method for optical sensing, the method comprising: - generating, at a first time instance, a light signal on the scene; - preparing at least one optical sensor including a plurality of sensing units; - imaging a scene on said at least one optical sensor; - a timing module preparation step of preparing a timing module including a plurality of timing means; - an associating step of associating each sensing unit or each group of said sensing units with one timing means; - by each timing means, between the first time instance and the detection of a first photon by the corresponding sensing unit; or between the detection of the first photon by the corresponding sensing unit and a second time instance; a timing value generating step of generating a first timing value representing a time of - reading the first timing value of said timing means only if the corresponding sensing unit has a positive detection state; - determining at least one first depth value based on said first timing values; It has.

[0027] Preferred embodiments of the third aspect of the present invention include one or suitable combinations of two or more of the following features: the method further comprises the step of scanning said light signal over said scene along a trajectory. The method further comprises a light spot position step of determining, by the optical sensor, a position of a light spot generated by the light signal on the scene, and a step of determining, by the optical sensor, at least one second depth value, the second depth value being calculated based on the position of the light spot by triangulation. More preferably, the second depth value is calculated based on the position of the light spot by triangulation of data from the light source and data from the optical sensor, or by triangulation of data from two optical sensors. - the method further includes the steps of outputting the first depth value at a predetermined distance or greater from the optical sensor and outputting the second depth value at a distance less than the predetermined distance from the optical sensor, the method further includes the steps of determining an initial estimate of the position of the light spot based on data from the sensing unit, and determining a final estimate of the position of the light spot based on data from the light source.

[0028] These and other characteristics, features and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the invention. [Brief explanation of the drawings]

[0029] The present disclosure is further explained by the following description and accompanying drawings. [Figure 1(a)] FIG. 1(a) is a diagram illustrating the steps by which a system (1) according to an embodiment of the present invention operates. [Figure 1(b)] FIG. 1(b) is a diagram illustrating the steps by which the system (1) according to an embodiment of the present invention operates. [Figure 1(c)] FIG. 1(c) is a diagram illustrating the steps by which the system (1) operates according to an embodiment of the present invention. [Figure 2]FIG. 2 shows an optical sensing system (1) including two optical sensors (2', 2'') in two different configurations (a, b) according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram illustrating triangulation according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating the filtering functionality of the system (1) according to an embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing the relationship between the distance (d) from the sensor (2', 2'') and the uncertainty or noise parameter (σz) according to an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram illustrating the operation of a time-to-analog converter present in a time-of-flight sensor according to an embodiment of the present invention. [Figure 7(a)] FIG. 7(a) illustrates a possible implementation of a system according to an embodiment of the present invention. [Figure 7(b)] FIG. 7(b) illustrates a possible implementation of a system according to an embodiment of the present invention. [Figure 8] FIG. 8 illustrates a possible implementation of a system according to an embodiment of the present invention. [Figure 9] FIG. 9 illustrates a possible implementation of a system according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating the filtering mechanism at the array level. DETAILED DESCRIPTION OF THE INVENTION

[0030] The present invention relates to an optical sensing system for three-dimensional imaging.

[0031] The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn to scale for illustrative purposes. The dimensions and relative dimensions do not correspond to actual reductions to practice of the invention.

[0032] Terms such as first, second, etc. in this specification and claims are used to distinguish between similar elements and are not necessarily used to describe an order in time, space, sequence, or otherwise. The terms so used are interchangeable under appropriate circumstances, and it is understood that the embodiments of the invention described herein are capable of operating in orders other than those described or illustrated herein.

[0033] Furthermore, terms such as top, under, and the like are used in this specification and claims for descriptive purposes and not necessarily to describe relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances, and that the embodiments of the invention described herein are capable of operation in orientations other than those described or illustrated herein.

[0034] Numerous specific details are set forth herein. However, it will be understood that embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this specification.

[0035] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, although they may. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[0036] Similarly, in describing exemplary embodiments of the invention, it should be understood that various features of the invention may be grouped together in a single embodiment, figure, or description for the purpose of streamlining the disclosure and facilitating an understanding of one or more various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single prior-disclosed embodiment. Accordingly, the claims following the detailed description are expressly incorporated into this specification, with each claim standing on its own as a separate embodiment of this invention.

[0037] Unless otherwise defined, all terms used in disclosing the present invention, including technical and scientific terms, have the meaning commonly understood by one of ordinary skill in the art to which this invention belongs. For further guidance, definitions of terms are included to better understand the teachings of the present invention.

[0038] As used herein, the following terms have the following meanings:

[0039] As used herein, "A," "an," and "the" refer to both the singular and the plural unless the context clearly dictates otherwise. By way of example, "a contaminant" refers to one or more contaminants.

[0040] The recitation of numerical ranges by endpoints includes not only the recited endpoints but also all numbers and fractions subsumed within that range.

[0041] In a first aspect, the present invention relates to an optical sensing system for three-dimensional (3D) imaging. The system comprises at least one optical sensor including a plurality of sensing units, each of which includes a photodetector. The optical sensor further comprises a timing module including a plurality of timing means. Each of the sensing units or each group of the sensing units corresponds to one of the timing means. The system further comprises at least one (e.g., pulsed) light source. For example, a light spot (or dot, semi-dot, or any other suitable shape) is generated by the light beam on a scene. The system further comprises an optical system capable of generating an image of the scene on the optical sensor.

[0042] The timing module is adapted to measure at least one first depth value, the first depth value being mathematically related to (or based on) the time between emission of a light pulse by a light source and receipt of the pulse by an optical sensor, i.e., by measuring time-of-flight. In other words, the first depth value may be calculated somewhere other than the timing module, but is calculated based on the output of the timing module.

[0043] The system further includes a read module adapted to read the first depth value from the timing means only if the sensing unit or sensing units corresponding to the timing means are assigned a positive detection state. For example, the system is adapted to filter the outputs of the sensing units, and the read module is adapted to read the first depth value from the timing means only if the filtered output of the sensing unit or sensing units corresponding to the timing means has a positive detection state. For example, a filtering mechanism (described elsewhere herein) exists that distinguishes between true positive and false positive detections. Thus, the read module is adapted to read the first depth value from the timing means when the sensing unit or sensing units corresponding to the timing means have a true positive detection and no false positive detection. It will be clear to those skilled in the art how to configure the components necessary to create the read module.

[0044] In other words, after the photodetector of the sensing unit detects one photon or a predetermined number of photons, for example, once the detection state becomes positive, the positive detection state is then evaluated as either a true positive state or a false positive state. If the sensing unit is assigned a true positive detection state, the reading module reads the time-of-flight measurement and the first depth value. In other words, the time-of-flight measurement is always performed when a photon is detected, i.e., for all positive detection states, but the time-of-flight information is read only when the sensing unit associated with the timing means is assigned a true positive detection state. This eliminates the need to read all values ​​from all timing means, and instead reads only the relevant values ​​associated with the true positive detection state, significantly reducing the power consumption of the system and dramatically increasing the speed at which measurements can be repeated. For example, in typical prior art LIDAR systems, the time-of-flight is measured for the entire imager, e.g., for each pixel of the LIDAR, and is also read for the entire imager, which is very power-intensive. However, in this case, only the time-of-flight corresponding to the relevant pixel (i.e., the pixel with a true detection) is read. Alternatively, one time-of-flight value may be measured for multiple pixels, e.g., a pixel and its neighboring pixels, with one time-of-flight measurement performed for each group of nine pixels. This does not significantly affect the measurement, since if the probability of detecting uncorrelated disturbance or thermal noise photons is sufficiently low compared to the probability of detecting an active photon for the pixel corresponding to the dot or laser beam position, the time-of-flight for the group of nine pixels is likely to be substantially the same, further reducing the power consumption of the system. Relying on a filtered output signal also improves the reliability of the system.

[0045] Each of the sensing units corresponds to one of the timing means. Alternatively, a group of the sensing units corresponds to one of the timing means. The former requires many timing means per optical sensor, while the latter requires fewer timing means per optical sensor. While the former may be more accurate, the latter may be advantageous for improving the detection uniformity of the timing means. This is because larger circuit components in the timing means allow those skilled in the art to reduce the effects of process variations and device inconsistencies. The latter (i.e., each group of sensing units corresponds to one timing means) is also advantageous in that it reduces the size of the optical sensor and improves its resolution. If each sensing unit corresponds to one timing means, it is difficult to obtain a high-resolution optical sensor. The latter is also advantageous in that it can operate at very high scene scanning frequencies, as will be explained later.

[0046] Preferably, the first depth value corresponds to either (i) a first timing value that is between a first time instance corresponding to the emission of light by the light source and the detection of a first photon by the corresponding sensing unit, or (ii) a second timing value that is between the detection of the first photon by the corresponding sensing unit and a second time instance (e.g., a clock signal) that is later than the first time instance. In the first case, the timing means start measuring when the emission of light from the light source starts, after which the detections are filtered as explained above, and only the relevant timing means corresponding to true positive detections are then read. On the other hand, in the second case, the timing means are adapted to start measuring the first depth value only after the corresponding sensing unit or units have a positive detection state, i.e., after filtering has taken place. The second scenario further reduces power consumption.

[0047] Preferably, the system is adapted to calculate at least one second depth value, wherein the second depth value is calculated by triangulation. The triangulation step can be performed on-chip (e.g., after transmitting position information from one sensor to another sensor, or after transmitting illumination / light-emitting element angle information to at least one sensor), or off-chip (e.g., after transmitting position information to a processing unit (such as a host system on-chip or a CPU)). For example, the sensing unit is adapted to determine the position of a dot in a scene, e.g., to search for events in the scene. For example, this can be done by detecting the position of the dot by an optical sensor and triangulating the position with data of the light source. Alternatively, this can be done by detecting the position of the dot by two optical sensors, e.g., two sensing units of two optical sensors, and then triangulating the position. For example, by triangulating the position (x,y) of the dot from two viewpoints, a depth value (i.e., a z value representing the distance between the optical sensor and the object in the scene) can be calculated, and thus the (x,y,z) coordinate of the position can be calculated at each time step. In this configuration, the light source is advantageous in that it allows triangulation and determination of the depth value of the field of view.

[0048] Because triangulation requires data from two viewpoints, it is susceptible to occlusions. For example, triangulation becomes impossible when only one viewpoint is available, such as when an object blocks the field of view of one optical sensor. However, measuring the time between laser generation and laser reception does not require two viewpoints. Therefore, combining the two techniques can reduce blind spots due to occlusions. For example, one embodiment of the present invention comprises a system including a first sensor and a first scanning illuminator positioned substantially close to each other to minimize parallax. In this manner, occlusion of the scanning illuminator due to parallax does not occur in time-of-flight-based measurements. The system further includes a second sensor and a second scanning illuminator positioned a design distance from the first sensor, creating a stereo pair of sensors that exhibits a baseline between them. The baseline generates parallax, which converts distance or depth into a parallax measurement (the difference in sensing position of the scanning dots on each sensor's image plane), which is used for triangulation, and depth is estimated by the second method (triangulation).

[0049] If the detection is correct, the two obtained depth values ​​are ideally equal or substantially equal. This provides an additional filtering mechanism. For example, considering the noise characteristics of each method, if the two obtained depth values ​​are not substantially equal, this may be a noisy or incorrect detection. Also, if triangulation fails due to a lack of data from two viewpoints, the first depth value can be used.

[0050] Having two depth values ​​allows flexibility in choosing a preferred depth value depending on the situation. For example, triangulation may be more accurate at short distances less than 10 times the baseline, while time-of-flight may be more accurate at long distances more than 10 times the baseline. Preferably, the system combines the first and second depth values ​​to provide a more reliable depth estimate, or alternates between the first and second depth values.

[0051] Preferably, triangulation can be preceded by a filtering step, for example to reduce the number of sensing units with false positive detections and triangulate only the data of sensing units with true positive detections, for example by checking the detections of neighboring sensing units (pixels) or by checking the persistence of detections over time. This is an intra-pixel filtering mechanism. Other filtering mechanisms are also possible, for example by projecting the detection values ​​to the surroundings and then filtering out noisy detection values ​​because the projection pattern is known (e.g., Lissajous).

[0052] Preferably, the system comprises scanning means adapted to scan the light beam from said light source along a trajectory, preferably continuously, over the scene, where scanning refers to sequentially illuminating different parts of the scene with said light source. It is worth noting that the grouping of sensing units into groups, in combination with one timing means corresponding to each of said groups, allows very fast scanning.

[0053] Scanning the light beam from the light source is advantageous in enabling triangulation. For example, using a system including a light source that projects a light beam onto a scene (e.g., environment) and two optical sensors, for example, positioned at different orientations relative to each other, where the two sensors have a shared field of view of the scene, it is possible to convert the xy-time data of the two sensors into xyz-time data by triangulation. For example, by adapting the light source to project a light beam onto the scene in an illumination trace, the light source includes a means adapted to scan the light beam over the scene (preferably continuously), and the optical sensor monitors the light spot generated by the light beam and outputs the position of at least one object point in the scene (e.g., a point on the surface of the object) along the trace at multiple instances, and the xy-time data of the two optical sensors can be converted into xyz-time data using triangulation. The light source, for example, serves as a reference point, and the position of the object point of a first optical sensor can be triangulated with the position of a second optical sensor to create a depth or z-dimension.

[0054] Preferably, the light source is adapted to output a dot or a dot-like pattern, and the scanning means scans the dot or dot-like pattern over the scene. For example, the light source is a point light source or any other light source capable of generating a dot or a dot-like pattern. This is advantageous for maximizing the signal-to-noise ratio, so that dots can be easily detected and reliable detection can be ensured. Dot illumination also reduces power consumption compared to other illumination types. Most preferably, the pattern is not a line pattern.

[0055] Preferably, the system is adapted to sequentially acquire depth data along the trajectory. In other words, the scanning and subsequent data acquisition are performed sequentially. For example, the dot is scanned along the trajectory, and at each time step, depth data (e.g., first and / or second depth values) corresponding to the position of the dot on the trajectory are acquired. This allows a scene to be perceived without having to scan the entire scene. For example, by scanning the scene only a few times, a large portion of the scene is scanned, thereby allowing an understanding of the scene to be obtained in a quick manner with less power.

[0056] Preferably, the timing means is adapted to reset when a sensing unit in a group of sensing units corresponding to said timing means has a false detection condition, thereby further reducing power consumption.

[0057] Preferably, the scanning means has an emission angle that changes very rapidly as the scanning means scans the scene. For example, the reflected light from the scanned beam traverses the sensor at a speed similar to the speed of a received pulse from a time-of-flight system. For example, the emission angle changes at a speed comparable to the time-of-flight, i.e., the time between when the light source emits a light pulse and when the sensing unit receives the pulse. For example, the scanning speed is substantially high so that the emission angle changes at a speed comparable to the time-of-flight. For example, the time between when the emission angle changes from one angle to another is comparable to the pulse length in pulsed operation. In other words, the present invention is not limited to pulsed operation but can be generalized to CW operation. This is because the scanning means scans the scene at such a fast rate that the emission angle changes so rapidly that it appears as if it were pulsed operation. For example, the time spent by the scanning means at each emission angle is substantially short, making it appear as if a pulse were emitted. For example, the scanning speed of the scanning means is at least 1 MHz, preferably at least 10 MHz, thereby changing the angular output of the beam at a rate relevant to time-of-flight detection. Scan speed is defined as the time it takes for the light beam to travel from one edge of the scene, specifically from one edge of the field to the other.

[0058] That is, the first depth value can be calculated based on the angle of the scanning means at the time of light emission and the elapsed time since then. For example, if the scanning means is scanning at a speed of 10 MHz, emits light at a light emission angle Θ1 at time t=t0, and the sensing unit receives the emitted light at time t=t1, the first depth is calculated based on the time of flight t1-t0.

[0059] Preferably, each timing means is substantially co-located with its corresponding sensing unit or units, thereby reducing the signal propagation delay from the associated sensing unit to the associated timing means and providing the most accurate measurements. However, the timing means could also be provided on the periphery of the optical sensor and connected to the sensing units by a signal bus.

[0060] The present invention allows for measuring the depth of an illuminated portion of a scene using two or more measurement methods. The first and second depth values ​​obtained from each method (time-of-flight and triangulation) are uncorrelated, and can therefore be used as a constraint for the filtering method. For example, considering the noise characteristics of each measurement, if the first and second depth values ​​differ significantly, this is likely to be an inaccurate detection.

[0061] The present invention allows one to take advantage of time-of-flight capabilities (e.g., improved depth resolution at longer distances and solving occlusion issues when combined with triangulation) without consuming the power required to read the time-of-flight sensors across the entire imager. For example, in a typical LIDAR sensor, there is an array of time-of-flight sensors, and because it is not known which pixels have true detections, the entire imager must be read, which consumes significant time and power. However, by using optical scanning and triangulation as described herein, one can first know which pixels have fired, and among those, which pixels have true detections (through a filtering mechanism), and then read the time-of-flight of only those pixels that have true detections, thereby reducing power consumption, improving detection speed, and providing very low latency and accurate detection.

[0062] Preferably, the system includes at least two optical sensors. For example, data from a sensing unit of one optical sensor, e.g., a sensing unit with a positive detection at one timestamp, is triangulated with data from a sensing unit of another optical sensor with a positive detection at the same timestamp (on-chip or off-chip). For example, the location of an active signal in a scene (i.e., the location where a light source is shining) is determined by one sensing unit in a first optical sensor and one sensing unit in a second optical sensor. The locations are triangulated to find a second depth value. In this case, triangulation is performed without needing the data of the light source.

[0063] Preferably, the system includes at least two light sources. For example, each of the light sources corresponds to one of the two optical sensors, and preferably each of the light sources is approximately co-located with its corresponding optical sensor (minimizing parallax). In another configuration, the system may include two scanning means, e.g., each scanning means corresponds to one light source, and each scanning means is approximately co-located with its corresponding optical sensor. In another configuration, the system includes one common light source, a beam splitter, and two scanning means, each scanning means is approximately co-located with its corresponding optical sensor. This allows time-of-flight measurements to be performed individually for each optical sensor, further reducing or eliminating blind spots due to occlusions. This is a further improvement over the case where only one light source and one optical sensor are available, in which case occlusions due to time-of-flight measurements are reduced but not completely eliminated.

[0064] Preferably, the timing means is a time-of-flight sensor including a time-to-analog converter (TAC). For example, the time-to-analog converter is adapted to output an analog value corresponding to the time-of-flight (i.e., the time between when the light source emits the light pulse and when the optical sensor receives the light pulse). For example, the time-to-analog converter includes a capacitive element that charges when the light source emits the light pulse and stops charging when the light pulse is received by the sensing unit. Advantageously, the analog value provides an accurate time-of-flight measurement. The time-to-analog converter may be replaced by a time-to-digital converter. However, a time-to-digital converter is less preferred because it requires a high-speed clock throughout the imager (i.e., throughout the optical sensor), which is energy consuming, especially when the imager is large in size.

[0065] Preferably, the TAC value is read only for pixels that have been assigned a true positive detection state, meaning that there is a high probability that an active light spot is located at those pixels at that time, and is read after filtering out ambient light using, for example, one of the filtering methods described in this document or previous patent applications PCT / EP2021 / 087594 and PCT / IB2022 / 058609. Filtering may also be based on the fact that the illumination is performed in a known pattern, for example a Lissajous pattern. A substantial part of the filtering may be applied before passing the photon detection to the timing means, meaning that the timing means is triggered or enabled by a filtered version of the photon detection signal.

[0066] For example, one TAC may be configured per pixel (i.e., sensing unit), or one TAC may be configured per group of pixels, such as every fourth pixel or every ninth pixel. For example, the TAC will be locked regardless of which pixel fires first. This is advantageous for reducing the number of TACs in the system. It also improves detection uniformity, as the larger TAC size provides uniformity. However, a balance may need to be struck, as having one TAC per group of pixels can introduce errors, for example, due to the TAC being locked by a pixel with a false detection.

[0067] To reduce the possibility of the TAC locking due to disturbance photon detection or thermal noise, the trigger signal going to the TAC / TDC may be filtered based on simultaneous firing of neighboring pixels, based on spatiotemporal constraints, or based on historical events or historical events from neighboring pixels, past firing, ... etc.

[0068] Preferably, the system is adapted to output the first depth value above a predetermined distance from the optical sensor, and the system is adapted to output the second depth value below the predetermined distance from the optical sensor. For example, the system is adapted to calculate a universal depth value, where above a predetermined distance from the optical sensor, the universal depth value is the first depth value, and below a predetermined distance from the optical sensor, the universal depth value is the second depth value. That is, the response of the system varies with distance. As will be shown below, triangulation and time-of-flight methods have different levels of uncertainty at different distances. For example, triangulation is most effective at short distances, while time-of-flight is most effective at long distances.

[0069] Alternatively, both the first depth value and the second depth value are acquired for all distances between the optical sensor and objects in the scene, i.e., for all objects in the scene regardless of their distance from the optical sensor. As described above, since both methods are uncorrelated, the first and second depth values ​​acquired by each method (time-of-flight and triangulation) can be imposed as constraints in the filtering method. For example, if the first and second depth values ​​are substantially different considering the noise characteristics of each measurement, a false positive is likely.

[0070] In a preferred embodiment of the present invention, the optical sensor has a low resolution, including a small number of sensing units, e.g., up to 1000 x 1000 pixels or up to 100 x 100 pixels. For example, the sensing units are adapted to calculate an initial estimate of the position of the light beam on the scene. Based on this initial estimate, the light source is adapted to obtain a final estimate of the position of the light beam on the scene, since the location where the light source was shining at each timestamp is known. For example, only a coarse x,y position of the dot (i.e., laser spot) can be obtained by each sensing unit. However, this may be sufficient, since the exact angular position or x,y position as seen from the light source is known from the light source. Typically, the exact x,y position is required to calculate the parallax from the difference between x,y and then the depth value. However, in this case, time-of-flight measurements are possible, and the first depth value can be calculated based on this. This allows for a smaller system and lower power consumption.

[0071] Preferably, the photodetector is a single-photon detector, preferably a single-photon avalanche detector. Alternatively, the photodetector is an avalanche photodetector. The use of SPADs in pixels is advantageous because they are sensitive to single photons. This means that the active projection structure requires minimal energy because the photodetector is sensitive to single photons. Another advantage is the sub-nanosecond response time, meaning that photons can be detected and encoded into a digital signal within nanoseconds.

[0072] Preferably, each photodetector is arranged in a reverse bias configuration. Preferably, the photodetector is capable of detecting a single photon incident thereon. Preferably, the photodetector is adapted to output a logic signal, e.g., an electrical detection signal, upon detection of a photon. For example, the detection signal can be represented by a signal including a logic "1," e.g., detection, while no detection signal can be represented by a signal including a logic "0," e.g., no detection. Alternatively, the detection signal can be represented by a pulse signal, e.g., transitioning from a logic "0" to a logic "1" and then back from a logic "1" to a logic "0." Also, no detection can be represented by or result in the absence of such a pulse signal.

[0073] Preferably, the scanning means is adapted to scan the light beam from the light source over the scene at least partially along a trajectory. This is advantageous in that it allows triangulation and the determination of depth values ​​within a field of view. For example, the depth values ​​can be estimated by triangulating the points of the at least one object in the scene detected by the optical sensor with data on the light emitted by the at least one light source. This is similar to triangulating the outputs of two optical sensors, since the data on the light emitted by the light source is known, i.e. it is known which part of the scene the light will illuminate at a given time.

[0074] Preferably, the system comprises at least one optical sensor, and distance is estimated by the displacement on the sensor of an optical signal corresponding to a point of the at least one object in the scene detected by the optical sensor, where the displacement is referred to as an expected position based on prior knowledge of a light source and / or the displacement is referred to as the displacement of an optical signal corresponding to a point of the at least one object in the scene detected by at least one other optical sensor, which is advantageous in that the distance between each object and the sensor can be easily determined.

[0075] The scanning is preferably substantially continuous so that objects in the scene are continuously scanned and identified. For example, a light source generates a light beam, which generates a light spot on the object. The light beam is continuously scanned over the scene along the trajectory. For example, the light beam scans all or almost all of the scene at any given time. The reflected signal is received by at least one optical sensor. The scanning may be, for example, a Lissajous pattern or a raster scan. For example, the scanning may be two-dimensional, e.g., a diagonal or diagonal-like pattern, or, instead of scanning horizontally or vertically, scanning both horizontally and vertically simultaneously, e.g., scanning in a direction other than along the row and column directions. This allows for faster scene perception. The scanning means may be, for example, a MEMS scanner, a mirror, an optical phased array, a metasurface approach beam scanning, etc. The light beam itself may be continuous or pulsed.

[0076] Preferably, the light source is adapted to a wavelength detectable by the optical sensor, for example between 100 nanometers and 10 micrometers, preferably between 100 nanometers and 1 micrometer. For example, the optical sensor is a photodetector or a matrix of photodetectors capable of detecting photons incident on each detector within a wavelength detection window in the range of 100 nanometers to 10 micrometers, preferably in the range of 100 nanometers to 1 micrometer.

[0077] Preferably, the system includes 100 or more pixel sensors, preferably 1,000 or more pixel sensors, more preferably 10,000 or more pixel sensors, even more preferably 100,000 or more pixel sensors, and most preferably 1,000,000 or more pixel sensors. For example, the system may be arranged in a matrix, with the optical sensors including rows of 1,000 pixel sensors and columns of 1,000 pixel sensors. Alternatively, the system may operate with a lower-resolution optical sensor, for example, including 100 x 100 pixels. For example, the light beam forms a dot on the scene, and the sensing unit is adapted to acquire the position (x, y) of the dot. However, due to the low resolution, only a coarse (x, y) position can be acquired. However, the precise (x, y) position can be acquired from the light source. In this case, triangulation may not be necessary, since a first depth value (obtained by time-of-flight measurement) is sufficient.

[0078] Preferably, the system comprises image representation means, for example a screen-like image representation device or other image representation device, for reconstructing the positions of points of said object in said scene.

[0079] Preferably, the optical system is used for 3D vision applications. For example, the system may be used to visualize objects in three dimensions. Alternatively, the sensor may allow for analyzing the scene, for example by extracting features of objects in the scene, without necessarily generating an image of the scene. Preferably, the system further comprises multiple optical sensors and / or multiple light sources. This is advantageous for creating 3D vision. For example, each optical sensor may be oriented differently so that a 3D perception of the imaged scene can be obtained, for example by triangulating the outputs of the two optical sensors.

[0080] In a second aspect, the present invention relates to an optical sensing system for three-dimensional imaging, the system comprising: - an optical sensor comprising a timing module including a plurality of sensing units (3) and a plurality of timing means, each of said sensing units (3) comprising a photodetector; - at least one light source; - scanning means, preferably adapted to scan a light beam from said light source over the scene along a trajectory; an optical system capable of generating an image of the scene on the optical sensor; It has. the timing module is adapted to measure at least one first depth value, the first depth value being based on a time between emitting a light pulse by the light source and receiving the pulse by the optical sensor; The optical sensor is adapted to calculate at least one second depth value, the second depth value being calculated by triangulation, preferably by triangulating data of the optical sensor with data of the light source, or by triangulating data of at least two optical sensors.

[0081] Any feature of the first aspect can be correspondingly described in the second aspect.

[0082] In a third aspect, the present invention relates to a method for optical sensing, the method comprising the steps of generating an optical signal on a scene at a first time instance, e.g., generating a laser spot or dot on the scene by a light source. The method further comprises providing at least one optical sensor including a plurality of sensing units, e.g., as disclosed in the first aspect. The method further comprises imaging the scene on the at least one optical sensor by at least one imaging optical system. For example, reflected light from the scene is received by at least one sensing unit of the plurality of sensing units of the at least one optical sensor. The method further comprises providing a timing module including a plurality of timing means. The method further comprises associating each sensing unit or each group of the sensing units with one timing means.

[0083] The method further comprises generating, by each timing means, a first timing value representing the time between (1) the first time instance and the detection of a first photon by a corresponding sensing unit, or (2) the detection of the first photon by the corresponding sensing unit and a second time instance (e.g. a clock edge) that is later than the first time instance. The second option is advantageous in reducing power consumption, as the timing means are activated only after the detection of a pulse, i.e. the timing means are not activated before the detection of a pulse.

[0084] The method further comprises filtering the outputs of the sensing units. The method further comprises reading out a first timing value of the timing means when the filtered output of a corresponding sensing unit (or group of units) has a positive detection state. The method further comprises determining at least one first depth value based on the first timing value, i.e., based on measuring the time between generating the light signal (e.g., by a light source) and receiving the pulse by the optical sensor.

[0085] Preferably, the method further comprises scanning the light signal (e.g. from the light source) along a trajectory, preferably continuously, over the scene. Preferably, the scanning has a scan rate (defined as the time it takes for a light beam to travel from one edge of the scene to the other) of at least 1 MHz, preferably at least 10 MHz. In other words, the method comprises varying the emission angle of the scanning means at such a fast rate that CW operation appears to be pulsed, as described in the first aspect.

[0086] Preferably, the method further comprises determining a position of a light spot generated by the light signal on the scene by the optical sensor. For example, at least one possible position of an active signal is determined by the optical sensor, i.e., by at least one sensing unit. The method further comprises determining at least one second depth value by the optical sensor. The second depth value is calculated based on the position of the dot by triangulation, preferably by triangulation of data from the light source and the optical sensor, or by triangulation of data from two optical sensors (e.g., two sensing units of two optical sensors).

[0087] Preferably, the method further comprises the steps of outputting the first depth value at a predetermined distance or greater from the optical sensor, and outputting the second depth value at a distance less than the predetermined distance from the optical sensor.

[0088] Preferably, the method further comprises determining an initial estimate of the position of the dots based on data from the sensing unit and determining a final estimate of the position of the dots based on data from the light source, for example the initial estimate being a coarse estimate based on a low resolution optical sensor and the final estimate being a fine estimate based on the position of the light beam.

[0089] In a fourth aspect, the present invention relates to a method for optical sensing, the method comprising: A light source generates a laser dot on the scene; - receiving, by at least one sensing unit of the plurality of sensing units of the at least one optical sensor, reflected light from the scene; - determining the position of the dots by the optical sensor; - determining by said optical sensor at least one second depth value based on the positions of the dots by triangulation, more preferably by triangulating data of said light source and data of said optical sensor, or by triangulating data of two optical sensors; - determining at least one first depth value based on measuring the time between generating a light pulse by a light source and receiving said light pulse by an optical sensor; It has.

[0090] Any feature of the fourth or third aspect (method) can be described as corresponding to the first and second aspects relating to the system.

[0091] In a fifth aspect, the present invention relates to the use of a system according to the first and second aspects and / or a method according to the third or fourth aspect for optical sensing.

[0092] FIG. 1 illustrates steps by which a system according to an embodiment of the present invention operates. First, an optical signal is generated. For example, a laser dot is generated by illuminating a scene with a laser light source. For example, a light pulse is illuminated onto the scene. After striking an object in the scene, the light is reflected onto two optical sensors. That is, the scene is imaged onto two optical sensors, each preferably consisting of an array of pixels (i.e., sensing units), and preferably, each pixel is a single-photon detector, preferably a SPAD. The light is detected by the optical sensors, where the optical pulses are converted into electrical signals or logic pulses. Each pulse is time-stamped relative to a reference clock. The detected signal is then filtered, for example, using time and neighborhood information. For example, filtering can be based on the expected pulse width, the expected dot projection size, and past and current detections of the pixel and its neighboring pixels. Part of the filtering method may be applied to the pulses before they are timestamped, and only the filtered pulses are processed by the timing unit (either the TDC or TAC). The filtered detection signals are then output to row and column signal buses, revealing their position (i.e., x, y coordinates). This position information from the multiple sensing units is processed to obtain filtered position information that likely indicates the true position of the laser beam on the image plane. This allows triangulation between the two optical sensors to determine a second depth value. Further filtering can be performed on the row and column bus information, for example, based on previous bus information to identify pixel locations where photons resulting from the laser beam's activation light irradiation are most likely to have been detected.

[0093] Each sensing unit is associated with a timing means, which calculates the time from laser generation to photon detection at the corresponding sensing unit, or, in FIG. 1(b), the time from photon detection to a reference signal. This allows for the determination of a first depth value. Because the first and second depth values ​​should be substantially identical, the above configuration allows for additional validation / filtering steps. However, this method is advantageous in situations where triangulation is not possible and therefore the second depth value cannot be obtained. In the case of an occlusion condition, for example, when the field of view of one sensor is blocked by an object, triangulation is not possible, but it is advantageous if the depth value can be obtained using time-of-flight measurements, i.e., the first depth value. A readout is performed only for timing means whose corresponding sensing unit has a positive detection state, and the first depth value is obtained based on that. Finally, the laser dot is moved to another position, and the same steps are performed again there. In this way, the entire scene is scanned. Scanning is preferably performed in a Lissajous fashion. This illumination trajectory is advantageous in that after a few illumination cycles, a significant portion of the image is already illuminated, allowing for efficient and fast image detection. Other illumination patterns are also possible.

[0094] Also, for two time-of-flight measurements belonging to two different optical sensors and corresponding to the same object, the two measurements will be substantially equal or equal at longer distances, but even at shorter distances the time-of-flight measurements can be corrected for differences in the optical path length from the light source to each optical sensor.

[0095] FIG. 1(c) shows a slightly different approach, where determining the first depth measurement is repeated N times, e.g., 8 or 10 times. This can be advantageous, for example, by averaging the measurements, to obtain a more accurate measurement. This can be achieved by two implementations. The first implementation shines light on a spot, takes a depth measurement, repeats this N times, then moves the light to the next spot (e.g., using a MEMS-like scanner), and repeats. The second implementation, instead of having a single light source and scanning it across the scene, has an array of light-emitting elements, such as VCSELs, where the VCSELs can be turned on at the spots where light is desired. For example, VCSEL #1 shines light, then a depth measurement is taken, and this is repeated N times, after which VCSEL #2 does the same, and so on.

[0096] FIG. 2(a) illustrates an optical sensing system (1) for optical sensing according to an embodiment of the present invention. The optical sensing system (1) includes two optical sensors (2', 2''), each of which includes multiple sensing units (3) and multiple time-of-flight units. Each of the sensing units (3) includes a photodetector, and the time-of-flight units are time-of-flight sensors (e.g., including TACs). The optical sensing system (1) further includes an optical system (14) capable of generating an image of a scene (8) on the sensors (2', 2''). The optical sensing system (1) also includes a light source (5) that emits a light beam that generates a light spot on the scene (4). While the light source (5) in FIG. 2(a) is shown as being between the two sensors (2', 2''), other configurations are possible. For example, each sensor (2', 2'') may have its own light source (5) fixed in the same position as the sensor (2', 2''), as shown in FIG. 2(b). The optical sensing system 1 further comprises scanning means 6 adapted to scan the light beam over the scene 8. For example, the scanning means 6 may be a reflector or a MEMS mirror capable of scanning the light beam over the scene 8, for example, successively over different parts of the scene 8. For example, each sensor 2', 2" may have its own scanning means 6. The light beam is scanned along an illumination trajectory, for example in a Lissajous fashion. Reflected signals from the scene 8 are then captured by the sensors 2', 2".

[0097] Furthermore, Figure 2(a) shows an object 9 blocking the field of view (12, 13) of sensors 2', 2''. In this case, object 9 blocks field of view (12) of sensor 2', while also blocking field of view (13) of sensor 2'. Thus, fields of view (12, 13) overlap. Object 9 casts a shadow or blind spot (10) for sensor 2'' and another blind spot (11) for sensor 2'. Without a time-of-flight unit, these blind spots (10, 11) are only visible to one sensing unit, making triangulation of the blind spots (10, 11) impossible. With a time-of-flight unit, depth measurements can be determined for the blind spots (10, 11). For example, in FIG. 2(a), the time-of-flight measurement unit of sensor 2'' can see blind spot 11, and the time-of-flight measurement unit of sensor 2' can see blind spot 10. For example, sensor 2'' sends a signal to blind spot 11, and the signal is reflected back to sensor 2'', and based on the time-of-flight, a depth measurement of blind spot 11 is determined. Similarly, a depth measurement of blind spot 10 is determined. Another preferred system configuration is shown in FIG. 2(b). In this case, a light source 5 and a scanning means 6 are provided at each sensor 2', 2''. In different situations, one or another configuration can be used to reduce blind spots and provide better depth detection. For example, having two light sources and two optical sensors can eliminate occlusion issues compared to FIG. 2(a). For example, each light source and its corresponding optical sensor can resolve occlusions in one of the blind spots (10, 11), while the other light source and its corresponding optical sensor can resolve the other blind spot. Also, having two time-of-flight sensors (i.e., one for each optical sensor) provides additional filtering advantages because the two TOF values ​​should be substantially similar. For example, a large difference between the TOF values ​​of the two sensors likely indicates an error.

[0098] A light source (5) projects simple structures, such as ellipsoidal or circular dots, onto a scene and scans the scene in a raster, Lissajous, or other pattern. Figure 3 illustrates how triangulation works in the context of the present invention. Each camera is adapted to track the position of the dot in each image plane and output a stream of (x, y, t) data, where x, y are coordinates in the image plane (pixel dimensions) and t is the timestamp of the detection. This has the advantage that such a stream can be output with a time resolution of up to 10 nanoseconds. For example, Figure 3 shows a laser-based illumination continuously sweeping the world in rapid strokes. The Lissajous pattern is generated based on a high-speed 2D MEMS mirror. Two or more sensors take snapshots of the laser dot's position at ultra-high speed (up to 100 MHz). Each sensor transmits the laser dot's position, and then a simple triangulation algorithm can calculate its precise 3D shape, position, contour, and motion at ultra-high speed. One advantage of this invention is the use of time-of-flight sensors in addition to triangulation, which solves some of the problems disclosed throughout this specification.

[0099] The system can filter false positives caused by thermal noise or ambient light incident on the sensor area. Figure 4 shows (a) the ground truth, (b) the raw detection, and (c) the filtered detection. The first filtering step occurs within the pixel plane, and this filter operates in parallel on each pixel. Typically, this filter represents either a spatial filtering method, a temporal filtering method, or a combination thereof. These filters can be implemented using transistor-level, RTL, more complex computational architectures, neural networks, etc.

[0100] FIG. 5 shows the relationship between the distance from the sensor (d) and the uncertainty or noise parameter (σ) for three different scenarios (a, b, c) according to an embodiment of the present invention. z) is shown. (a) shows the relationship between distance and uncertainty for a time-of-flight sensor (15) and for triangulation using a high spatial resolution pixel array (16) and a low spatial resolution pixel array (17). As can be seen, the performance of the time-of-flight sensor is better (i.e., the uncertainty is smaller) at longer distances. On the other hand, triangulation is best at short distances.

[0101] In triangulation, the noise parameter increases with distance because the disparity measurement is quantized. This is because the disparity decreases with distance, and at some point, the disparity becomes smaller than the pixel size, resulting in inaccurate detection. Those skilled in the art will recognize that distance is inversely proportional to disparity. Therefore, triangulation measurements are inaccurate for long distances (e.g., more than 10 times the baseline). For example, when the disparity is smaller than a pixel, the distance cannot be measured. However, triangulation measurements are very accurate at short distances (e.g., less than 10 times the baseline) due to the large disparity. On the other hand, time-of-flight (TOF) is better at longer distances because it is less susceptible to parallax issues. However, it performs worse at short distances due to diminishing returns from improving the signal-to-noise ratio. Therefore, the value of the y-intercept is the spatial resolution (23) of the time-to-flight sensor's time-to-analog converter (TAC). Therefore, the present invention preferably relies on using time-of-flight sensors at longer ranges and / or in case of occlusions, and on using triangulation at longer ranges.

[0102] Figure 5(b) shows the results of combining the responses of (15) and (16) to obtain the first hybrid response (18), which shows excellent performance at both long and short distances. Similarly, Figure 5(c) shows the results of combining the responses of (15) and (17) to obtain the second hybrid response (19). The responses (15, 16) depend on the resolution and baseline of the sensor (i.e., pixel array). For applications requiring a high-resolution pixel array, the first hybrid response (18) is desirable. However, for other applications, the second hybrid response (19), which enables very compact systems, such as a 100x100 pixel array, is also useful. Lower-resolution pixel arrays are cheaper and consume less power. As mentioned in the discussion, such low-resolution pixel arrays result in coarse x- and y-locations of the dots, but this may be sufficient because the exact location of the dots is known from the light source. In this case, parallax measurements are not necessary because depth values ​​can be obtained from the time of flight.

[0103] Figure 6 illustrates the operation of a time-to-analog converter in a time-of-flight sensor according to an embodiment of the present invention. First, a pulse (20) is transmitted from the light source (5) at the falling edge (21) of the clock signal (24), which is reflected and received (22) by the time-of-flight sensor. The time between the transmission and reception of the pulse is the time-of-flight (ToF). To calculate this time, an integrating capacitor begins charging when the pulse is transmitted (20) and stops charging (i.e., holds its charge) when the pulse is received (22). The time-of-flight is calculated by dividing the analog value (P a ) Finally, after two clock cycles the capacitor is read, then discharged and reset to its initial value. This information can be converted using an analog-to-digital converter. For example, detection and triangulation can be done in the first clock cycle, and then the capacitor value is read at the second or end of the second clock cycle. It will be clear to those skilled in the art that many other implementations are possible.

[0104] Figure 7(a) shows a possible implementation of the system of the present invention. A pixel has a detection unit therein, which in turn has a photodetector. The photodetector triggers upon the detection of a photon. The output of the sensing unit is filtered using a spatiotemporal filter, with the assistance of neighboring sensing units (i.e., checking whether neighboring sensing units are also triggering). The filter's output is passed to a detection status flag, which indicates whether the detection is a true positive or a false positive, for example, based on previous information. This output may also trigger a timing means if the delay introduced by the filtering is minimal or systematic. This behavior is intended to avoid triggering the timing means with every photon detection, regardless of local filtering or positive detection status evaluation. The readout means then reads out the timing means based on whether the sensing unit is associated with active light with a high probability. Figure 7(b) shows the same implementation, but with clusters of pixels, each sharing a single timing means.

[0105] A possible implementation of the system of the invention is shown in Figure 8. This implementation is advantageous when the filter adds delay to the timing means that is undesirable, i.e. that causes inaccurate measurements in the timing means.

[0106] Figure 9 shows a possible implementation of the system of the present invention. Compared to previous implementations, several differences can be seen. First, there is an evaluation logic module, which triggers the readout means to read out the timing means if this is a true positive detection. Second, the evaluation logic can perform additional filtering to ensure that this is a true positive detection. While the previous filtering steps were at the pixel level, for example, by knowing the position of the illuminating light, the evaluation logic can perform additional filtering at the array level, as shown in Figure 10.

[0107] Other arrangements for achieving the objectives of the methods and apparatus embodying the present invention will be apparent to those skilled in the art. Details of specific embodiments of the present invention will now be described. However, no matter how detailed the above description may appear in text, it will be apparent that the present invention can be applied in many ways. It should be noted that the use of a particular term in describing a particular feature or aspect of the present invention should not be construed as meaning that the term herein be redefined so as to be limited to the particular feature or aspect of the invention with which the term is associated. [Explanation of symbols]

[0108] 1. Optical Sensing System 2', 2'' two optical sensors 3 Sensing Unit 5 light source 6. Scanning means 8 Scenes 9 Shield 10 2" blind spot 11 2' blind spot 12.2" field of view 13 2' field of view 14 Optical system 15 Relationship between distance and uncertainty of time-of-flight sensors 16 Relationship between distance and uncertainty of high-resolution pixel array 17 Relationship between distance and uncertainty of low-resolution pixel array 18 First Hybrid Response 19 Second Hybrid Response 20 Radiation Pulse 21 Falling edge of the clock signal 22 Received pulse 23 TAC resolution 24 clock signals

Claims

1. An optical sensing system (1) for three-dimensional imaging, comprising: at least one optical sensor (2') including a timing module including a plurality of sensing units (3) and a plurality of timing means, each of the sensing units (3) including a photodetector, each of the sensing units (3) corresponding to one of the timing means, or the sensing units (3) being divided into groups, each group of the sensing units (3) corresponding to one of the timing means; at least one pulsed light source (5); an optical system (14) capable of generating an image of the scene (8) on said optical sensor (2'); a reading module; and each timing means adapted to measure at least one first depth value, said first depth value being mathematically related to the time between emission of a light pulse by said light source (5) and reception of said light pulse by said sensing unit (3); the optical sensing system (1) is adapted to filter false positives caused by thermal noise and ambient light from the output of the sensing unit (3); the reading module is adapted to read the first depth value from the timing means only if the filtered output of the sensing unit (3) corresponding to the timing means or the filtered output of the group of sensing units (3) corresponding to the timing means has a positive detection state; Optical sensing system (1).

2. 2. The optical sensing system (1) according to claim 1, adapted to calculate at least one second depth value; the second depth value is calculated based on the position of the light spot of the light source (5) by triangulating data of the optical sensor (2') with data of the light source (5) or by triangulating data of at least two optical sensors (2', 2''). Optical sensing system (1).

3. 3. An optical sensing system (1) according to claim 1 or 2, and a scanning means (6) adapted to scan the light beam from the light source (5) over the scene (8) along a trajectory. Optical sensing system (1).

4. 4. The optical sensing system (1) according to claim 3, the light source (5) is adapted to output a dot or a dot-like pattern, and the scanning means (6) scans the dot or dot-like pattern over the scene (8); Optical sensing system (1).

5. 5. The optical sensor according to claim 3, The optical sensing system (1) comprises: adapted to sequentially obtain the first and / or second depth values ​​along the trajectory. Optical sensor.

6. An optical sensing system (1) according to any one of claims 3 to 5, The scanning means (6) has a scanning speed, the scanning speed is the time it takes for the light beam to move from one edge of the scene (8) to the other, the scanning speed being at least 1 MHz, preferably at least 10 MHz; Optical sensing system (1).

7. An optical sensing system (1) according to any one of claims 1 to 6, each timing means being substantially co-located with its corresponding sensing unit (3) or sensing units (3); Optical sensing system (1).

8. An optical sensing system (1) according to any one of claims 1 to 7, the optical sensing system (1) comprises at least two optical sensors (2'') and preferably at least two light sources (5); each of the light sources (5) corresponds to one of the two optical sensors (2', 2''); preferably, each of the light sources (5) is at the same position as the corresponding optical sensor (2', 2''); Optical sensing system (1).

9. An optical sensing system (1) according to any one of claims 1 to 8, The first depth value is a first timing value between a first time instance corresponding to an emission of light by the light source (5) and a corresponding detection of a first photon by the sensing unit (3); or a second timing value between the detection of the first photon by the corresponding sensing unit (3) and a second time instance that is later than the first time instance; It corresponds to either Optical sensing system (1).

10. An optical sensing system (1) according to any one of claims 1 to 9, the timing means is a time-of-flight sensor including a time-to-analog converter adapted to output an analog value corresponding to the time between emission of the light pulse by the light source (5) and reception of the light pulse by the optical sensor (2'); Optical sensing system (1).

11. An optical sensing system (1) according to any one of claims 2 to 10, the optical sensing system (1) is adapted to output the first depth value at a predetermined distance or greater from the optical sensor (2'), and to output the second depth value at a distance less than the predetermined distance from the optical sensor (2'); Optical sensing system (1).

12. An optical sensing system (1) according to any one of claims 2 to 10, both the first depth value and the second depth value are obtained for all distances between the optical sensor (2', 2'') and objects in the scene (8); Optical sensing system (1).

13. An optical sensing system (1) according to any one of claims 2 to 12, the optical sensing system (1) is adapted to determine an initial estimated area within the scene (8) in which the light spot is located, and to determine a final estimated area within the scene (8) in which the light spot is located based on a position where the light source (5) is shining, the final estimated area being smaller than the initial estimated area; Optical sensing system (1).

14. 1. A three-dimensional imaging method, comprising: a light spot generation step of generating a light spot on the scene (8) at a first time instance; an optical sensor preparation step of preparing at least one optical sensor (2') including a plurality of sensing units (3); an imaging step of imaging a scene (8) on said at least one optical sensor (2'); a timing module preparation step of preparing a timing module including a plurality of timing means; an associating step of associating each sensing unit (3) with one timing means or dividing the sensing units (3) into groups, each group of sensing units (3) being associated with one timing means; By each timing means, between said first time instance and the corresponding detection of a first photon by said sensing unit (3); or Between the detection of the first photon by the corresponding sensing unit (3) and a second time instance that is later than the first time instance; a timing value generating step of generating a first timing value representing a time of a filtering step for filtering the output of the sensing unit (3); a reading step of reading the first timing value of the timing means only if the filtered output of the corresponding sensing unit (3) or the filtered output of the corresponding group of sensing units (3) has a positive detection state; determining at least one first depth value based on the first timing value; having Three-dimensional imaging methods.

15. 15. The three-dimensional imaging method according to claim 14, determining the position of said light spot on said scene (8) by said optical sensor (2'); a second depth value determination step in which at least one second depth value is determined by the optical sensor (2') based on the position of the light spot by triangulation, more preferably by triangulation of data from the light source (5) and the optical sensor (2') or by triangulation of data from two optical sensors (2', 2''); further comprising Three-dimensional imaging methods.

Images