Ranging sensor array dynamic gating

Abstract

A method, system, and computer program for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections. At least one histogram of reflection arrival times counts of at least one pixel of a frame is analyzed. At least one metric for the at least one pixel is determined based on analysis on the at least histogram. Reflection collection for the at least one pixel is selectively disabled according to the at least one metric and a collection policy.

Description

[0001] RANGING SENSOR ARRAY DYNAMIC GATING

[0002] BACKGROUND

[0003] Some embodiments described in the present disclosure relate to range detection and, more specifically, but not exclusively, to dynamic gating of a pixel array in a ranging sensor.

[0004] Ranging sensors and / or ranging imaging systems are used in various applications such as autonomous vehicles and driver-assistance systems. Ranging sensors can detect the distance between themselves and an object without physical contact. There are several types of ranging sensors, including sonic sensors, visible or infrared light-based sensors, and time-of-flight (ToF) sensors. Sonic sensors use sound waves to detect obstacles, while visible or infrared light-based sensors use light to measure distances. Ranging imaging systems often use a combination of ranging sensors and cameras to create 3D images of the environment.

[0005] Light Direction and Range (LiDAR) is a technique for producing three-dimensional (3D) depth maps. It is analogous to Radio Detection and Ranging (RADAR), but uses visible light rather than radio waves. Direct Time-of-Flight (dToF) is a type of LiDAR, which works by precisely measuring the time difference between emitting a light pulse and receiving its reflection. Using appropriate optics, the distance of scene objects from the ranging sensor can be separately measured at each point in a two-dimensional (2D) image (i.e., a pixel array), resulting in a 3D depth map. A laser device is typically used to generate the brief, intense light pulse, which may be of the order of a nanosecond in duration.

[0006] To detect the reflected photons, a light sensor with excellent sensitivity must be used. This is typically a Single Photon Avalanching Diode (SPAD). An array of SPADs (typically 2D array) can be created to form a depth sensor. After a photon has struck a SPAD, and thus triggered an avalanche, there is a dead time (a period when the SPAD is insensitive to further photons until it is recharged).

[0007] The reflected photons strike the sensor at a fixed delay after the laser pulse, with the time delay determined by the distance to the object. SPADs may be time-gated, i.e. switched on only for a brief period when the reflection is expected to arrive. In addition to the received photons, there is typically some background light (noise) that arrives at random times. To correctly distinguish the true reflection from the background noise, the measurement is repeated many times, and the arrival times plotted in a histogram, with time delay on the x axis (horizontal direction) and number of photons on they axis (vertical direction). The peak in the histogram corresponds to the desired signal, i.e. the time of the reflection from the object.

[0008] Depth sensors may include a SPAD array manufactured on a silicon substrate. Lithography refers to the processes used to etch the detailed multi-layered integrated design onto the silicon wafer.

[0009] The intensity of light striking a surface is often measured in units of lux, which is defined as one lumen per square meter. The lumen is a unit of luminous flux, i.e. the amount of light emitted by a light source in all directions.

[0010] Reflected light is received as individual photons. When photons are continuously received at a low rate, this can be described as a Poisson process with Poisson statistics, i.e. the number of photons in a given time window may fluctuate due to the randomness of the process. This effect, which adds some uncertainty when trying to determine the intensity of the received light, is described as shot noise.

[0011] Autonomous moving devices typically need to track their location. Simultaneous Localization and Mapping (SLAM) is a technique to build a 3D digital map of the environment, while simultaneously keeping track of the current position within that environment.

[0012] SUMMARY

[0013] It is an object of the present disclosure to describe a system and a method for dynamic gating of a ranging sensor array.

[0014] The foregoing and other objects are achieved by the features of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures. According to an aspect of some embodiments of the disclosed subject matter there is provided a method for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the method comprising: analyzing at least one histogram of reflection arrival times counts of at least one pixel of a frame; determining at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disabling reflections collection for the at least one pixel according to the at least one metric and a collection policy.

[0015] According to another aspect of some embodiments of the disclosed subject matter there is provided a processing circuitry for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the processing circuitry comprising a memory and is configured to: analyze at least one histogram of reflection arrival times counts of at least one pixel of a frame; determine at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disable reflections collection for the at least one pixel according to the at least one metric and a collection policy.

[0016] According to yet another aspect of some embodiments of the disclosed subject matter there is provided a computer program for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the computer program comprising program instructions which, when executed by at least one processor, cause the at least one processor to: analyze at least one histogram of reflection arrival times counts of at least one pixel of a frame; determine at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disable reflection collection for the at least one pixel according to the at least one metric and a collection policy.

[0017] Optionally, analysis of the at least one histogram comprises calculating at least one statistic from counts of the at least one histogram.

[0018] Optionally, calculating the at least one statistic comprises calculation of at least one of: at least one peak value of counts of the at least one histogram; and a cumulative sum of values of counts of the at least one histogram.

[0019] Optionally, the at least one metric is computed from the at least one statistic, and wherein the at least one metric comprises at least one of energy consumption, confidence level, accuracy estimate, and detection duration estimate.

[0020] Optionally, for a pixel of the frame a number of transmissions by a signal source of the ranging sensor corresponding to activation timings of the pixel is counted, and the at least one metric is computed from the at least one statistic according to the number.

[0021] Optionally, the at least one histogram comprises a plurality of histograms of neighboring pixels.

[0022] Optionally, the collection policy is varied according to at least one region of interest.

[0023] Optionally, the at least one region of interest is determined through analysis on at least one of a depth map and an intensity image of at least one other frame preceding to the frame.

[0024] Optionally, the at least one region of interest is determined through analysis on scene information of an intensity image generated from histograms of at least one subset of pixels.

[0025] Optionally, the analysis on scene information of the intensity image is performed using a simultaneous localization and mapping, SLAM, process.

[0026] Optionally, for a pixel of the frame a number of transmissions by a signal source of the ranging sensor corresponding to activation timings of the pixel is counted, and a count value obtained as a function of a count in a histogram of the pixel is adjusted using the number.

[0027] Optionally, the count value is adjusted according to a ratio between the number of transmissions counted for the pixel and a total number of transmissions by the signal source for the frame.

[0028] Optionally, a signal source of the ranging sensor is directional, and a respective beam portion of the signal source at a direction corresponding to the at least one pixel is selectively disabled according to the at least one metric and the collection policy.

[0029] Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments, exemplary methods and / or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.

[0030] BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING, S )

[0031] Some embodiments are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments may be practiced.

[0032] In the drawings:

[0033] FIG. 1 is a schematic illustration of an exemplary direct time-of-flight (dToF) light direction and range (LiDAR) imaging system, according to some embodiments;

[0034] FIG. 2 is an exemplary histogram of photon-event times for one pixel in a dToF LiDAR imaging system, according to some embodiments;

[0035] FIG. 3 is a schematic illustration of an exemplary adverse scenario of high-power consumption in a dToF LiDAR imaging system, according to some embodiments;

[0036] FIG. 4 is another exemplary histogram of photon-event times for one pixel in a dToF LiDAR imaging system as obtained in an exemplary adverse scenario of a dark pixel, according to some embodiments;

[0037] FIG. 5 is a schematic illustration of an exemplary use case of a depth sensor in an automobile for parking assistance, according to some embodiments;

[0038] FIG. 6 is a schematic illustration of respective exemplary depth map and intensity image produced by a depth sensor, according to some embodiments;

[0039] FIG. 7 is a block diagram of an exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system, according to some embodiments;

[0040] FIG. 8 is a block diagram of another exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system using pixels block data analysis, according to some embodiments;

[0041] FIG. 9 is a schematic illustration of combining histogram data from multiple pixels of a sensor array, according to some embodiments;

[0042] FIG. 10 is a schematic illustration of pixels block grouping in a sensor array, according to some embodiments; and

[0043] FIG. 11 is a block diagram of yet another exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system using previous frame(s) data analysis, according to some embodiments.

[0044] DETAILED DESCRIPTION

[0045] Some embodiments described in the present disclosure relate to range detection and, more specifically, but not exclusively, to dynamic gating of a pixel array in a ranging sensor.

[0046] Three-dimensional (3D) direct time-of-flight (dToF) light direction and range (LiDAR) imaging systems, as well as other likewise ranging sensors and / or ranging imaging systems, use laser pulses to measure distance. Transmitted short (of order about 1 nanosecond(s) duration and / or the like) laser pulses are reflected by objects, and these reflections return to the sensor at different times. Optics are used to focus the image onto a sensor array, such as a two-dimensional (2D) array of Single Photon Avalanching Diodes (SPADs).

[0047] Reference is now made to FIG. 1 which is a schematic illustration of an exemplary direct time-of-flight (dToF) light direction and range (LiDAR) imaging system, according to some embodiments. As shown on FIG. 1, a 3D dToF LiDAR imaging system may use a laser emitter and / or likewise device adapted to produce short pulses of electromagnetic radiation, e.g. light flashes, as a source of signal transmissions. A two-dimensional (2D) pixel array of light sensitive sensors, such as a SPAD pixel array, may be used to detect photons arriving at the system, and the system may further record their arrival times, respective of when laser pulses were transmitted by the laser source. After laser pulses are transmitted, the reflections of the transmitted laser pulses, as returned from objects in the scene and / or environment to be imaged, may then travel back towards the SPAD pixel array. The reflections may be collected and directed by suitable optics towards respective pixels of the SPAD pixel array. The reflections from objects of the scene may arrive at the SPAD pixel array at different times, according to a length of travel by a laser pulse from the laser source to a respective object and back to the SPAD pixel array. For example, a travel back and forth of a laser pulse to and from a distant object is longer than that to and from a proximate object, such as illustrated in FIG. 1 with the cone-shaped and spherical objects, thus reflections from closer objects arrive earlier to the SPAD pixel array than from farther objects, as the speed of light is constant.

[0048] The array of SPADs in the sensor are triggered when returning photons strike them. The precise timestamps of these events are recorded. This process is repeated many times, and the timestamps built up into a histogram of photon-event times for each pixel. This gives a 3D volume (height, width and time) of reflected photons. The peak(s) in each histogram show the distance (depth) of the reflective surface seen by the corresponding pixel.

[0049] Reference is now made to FIG. 2 which is an exemplary histogram of photon-event times for one pixel in a dToF LiDAR imaging system, according to some embodiments.

[0050] As shown on FIG. 2, photon event times, i.e., recorded arrival times of photon reflections detected by a single element in the SPAD pixel array, may be categorized into bins, each of which being of a predetermined interval at width, e.g. 50 picoseconds and / or the like, together forming a domain of successive arrival times of photon reflections. All instances falling into a same bin may be counted, and the total number for each of the respective bins may be recorded, resulting in a histogram of photon event times (reflections arrival times). A peak value of the histogram, i.e., a maximum count of photon event instances over all bins, may be located, and its position (i.e., a timing of arrival denoted by the respective bin at which the events count is maximal) may be used to determine a depth value at the respective pixel.

[0051] One technical challenge of dToF LiDAR imaging systems, and / or of other likewise ranging sensors, is that in order to store a complete histogram for every pixel, there may be required a large amount of memory within the sensor, with the memory requirement increasing proportional to the depth resolution (i.e., width of each bin) and maximum depth range (i.e., how many bins are used in total). As only the “correct” depth is required (i.e., the position of the peak in the histogram), the histogram stores a large amount of data that is not needed.

[0052] As a result, many systems offer a choice between coarse depth measurement (where the depth resolution is poor) or stepped / windowed measurement. In stepped / windowed measurement, the histogram is configured to record only a subset of timestamps (e.g., between 0 nanoseconds and 10 nanoseconds, and / or the like). This measurement is repeated many times at different offsets, to gradually build a complete high-resolution histogram.

[0053] Another technical challenge of dToF LiDAR imaging systems and / or likewise ranging sensors is that there is a huge variation in scene dynamic range, and in the signal to background noise ratio (SBR).

[0054] The high dynamic range and / or variable SBR may be caused by one or more of the reasons as follows:

[0055] - Varying background light (unwanted noise): the 3D sensor may be used indoors and outdoors, where there may be an enormous range of background light intensity (e.g., as high as 100,000 lux in direct sunlight, on one hand, and down to less than 1 lux in darkness, on the other hand);

[0056] - Varying return signal: the strength of the laser return may be very strong (for a close, reflective object) or weak (a distant, small, unreflective object). This is expressed on the radar range equation (see equation (1) herein) which applies similarly to LiDAR detection as well. Very weak returns may be just a few photons, which are subject to Poisson statistics (shot noise).

[0057] Equation (1) herein describes the radar range equation, where the received power is proportional to 1 / R4(for object smaller than one pixel):

[0058] In the exemplary adverse scenario as shown in FIG. 3, a SPAD pixel array of a dToF LiDAR imaging system is exposed to direct sunlight over substantial amount of its area, thus resulting in excessively high energy consumption caused by multiple repeated photon events.

[0059] Each avalanche (photon event) may require energy of order between about 1 to 100 picojoule (pJ) - and they may occur continuously (at an interval of order of about 1 nanosecond per pixel) - causing energy consumption of order of about 1 milliwatt (mW) per pixel.

[0060] It should be noted that it is known that SPADs can be “time-gated” to enable them only at certain time periods, e.g. when a reflection photon is expected to arrive. This may be done to save power or to ensure the SPAD is sensitive to the peak (i.e. not in its post-avalanche “dead time” when it cannot detect photons).

[0061] However, time-gating may not eliminate entirely such excessive and / or high energy consumption, e.g., in case of very bright and / or over saturated pixels, such as in the exemplary adverse scenario of exposure to direct sunlight and / or the like as discussed and illustrated herein.

[0062] Another technical challenge resulting from the high dynamic range is poor histogram peak detection in dark pixels, such as for example pixels of non-reflective and / or distant surfaces, as there are very few photons (and it is therefore hard to determine where the peak lies).

[0063] Reference is now made to FIG. 4 which is an exemplary histogram of photon-event times for one pixel in a dToF LiDAR imaging system as obtained in such exemplary adverse scenario of a dark pixel.

[0064] As shown on FIG. 4, for a dark and / or under exposed pixel, such as may be the case for a non-reflective and / or distant surface, there may be a very low number overall of photon arrival events at the SPAD pixel array, and moreover, due to high variance of background noise as discussed herein, a variety of recorded arrival times of such sporadic photon events may be scattered across different and greatly disparate time bins of the histogram, where even those histogram bins populated by photon events may contain only a few such photons (e.g., merely one or two at most), and several candidate peaks at various bin positions may emerge, such that it may be hard if not impossible to tell which is the “correct” one representing the actual distance to that surface.

[0065] Pre-existing approaches for dToF LiDAR imaging tools and / or techniques provide for some of the characteristics as described in the following:

[0066] - One system offers the user a configuration choice between operating modes: either low-resolution mode or high- resolution mode. This refers to the 2D spatial resolution of the sensor. The low-resolution mode has better performance at long range, whereas the high-resolution mode picks up more detail at close range.

[0067] - Some systems offer a “stepped” or windowed histogram capture mode, which provides high resolution by repeating the depth measurement many times for different depth windows. For example, the first measurement might cover the range between 0 centimeters (cm) to 10 cm, the second measurement might cover the range between 10 cm to 20 cm, and so on. This approach allows the systems of this sort to provide high spatial resolution within a lower memory requirement.

[0068] - Some systems offer a “coarse” and “fine” depth measurement mode, which can be operated in a two-stage manner. Firstly, there is performed a coarse (low depth resolution) measurement to detect an approximate depth (i.e., histogram peak), followed by a second fine (high depth resolution) measurement covering the approximate depth indicated by the first measurement. For example, the first (coarse) mode might have a depth resolution of 10 cm, while the second (fine) mode might have a depth resolution of 1 millimeter (mm).

[0069] Such pre-existing techniques and / or tools however suffer from several disadvantages as described herein.

[0070] The system offering alternative “fine” and “coarse” 2D spatial resolution modes cannot adapt to the situation as it is being used. It must be configured either for long-range or for high-2D-resolution mode by the user. This means that, in many environments, either spatial resolution is sacrificed, or the depth result is noisy and unreliable. As typical automotive scenes include both near and far objects, it is not possible to configure this system for best performance on both objects.

[0071] Systems that offer a “stepped” or windowed histogram capture mode have a very slow frame rate, because they must repeat the depth measurement many times to collect different portions of the full histogram. This means that they also suffer from high power consumption per frame.

[0072] Systems that offer a two-stage depth measurement (i.e. coarse followed by fine) suffer from poor performance when SBR is low. The coarse and the fine stage must both successfully identify the correct peak to produce a correct depth measurement; if the correct peak is not identified in the coarse stage then the fine stage will operate at the wrong depth.

[0073] All pre-existing systems and / or methods also suffer from a power consumption problem under high background illumination (e.g. when the sun is visible). The SPADs in that part of the image are repeatedly reactivated, causing high power consumption.

[0074] The pre-existing systems and / or methods also perform the same number of depth measurements in all parts of the image, so they keep all the SPADs activated for the same total time (during same number of laser flashes). This makes it difficult to optimize the number of laser flashes, because a low number will cause distant objects to be undetectable (due to shot noise from weak reflections) whereas a high number will waste power measuring close objects (which give strong reflections).

[0075] The disclosed subject matter is aimed at improving upon, providing advantages over, and / or overcoming many shortcomings of pre-existing tools and / or techniques for range detection and / or imaging.

[0076] Firstly, the disclosed subject matter is aimed resolving or mitigating the adverse effect of high power consumption when there are excessively bright light sources visible (e.g. the sun, car headlights). This undesired result occurs because SPADs are continuously retriggered as new photons strike them, and each time the SPAD fires it consumes a portion of energy, at magnitude of about several picojoules (pJ). Due to the number of photons in bright conditions (and the number of pixels in a large sensor array), this sums to an excessive power consumption.

[0077] Secondly, the disclosed subject matter is aimed at dealing with the problem of poor peak detection in poorly reflective parts of the image, which leads to missed objects. This problem occurs because there are very few photons in these parts of the image, so the photon-arrival-time histogram for each pixel has high shot noise (because each bin samples the Poisson distribution using a very small ).

[0078] Thirdly, the disclosed subject matter addresses the problem of inconsistent depth accuracy across the image. This problem occurs because nearby, reflective objects reflect more photons back to the sensor than distant, nomeflective objects; this entails that the histogram peak is more clearly defined in the nearby object and can therefore be measured with greater accuracy than the distant object.

[0079] As used herein, the term “gating” refers to an operation scheme in which sensory elements or detectors are activated or enabled for only a part of a measurement or measuring cycle and deactivated at the rest of it in a selective manner, such as for example, “time gating” where a sensing device is activated only at specified times when signals are expected to arrive.

[0080] As used herein, the term “dynamic gating” refers to selective disabling and / or deactivation of sensory elements, e.g., pixel(s) in a sensor array of a ranging sensor or ranging imaging device, during a measurement or measuring cycle (e.g., capturing or reconstruction of an individual frame), in accordance to determinations made upon information gathered and / or updated at least in part while the measurement is still taking place, e.g., after each time a sensing signal is emitted from a source, such as a laser flash in a dToF LiDAR imager, and / or the like.

[0081] According to some embodiments, histograms of photon arrival times for each pixel are monitored throughout a measurement as they are progressively being built, and their contents analyzed to determine one or more pixel-related metrics, including but not limited to, for example: (i) the energy consumption metrics for each pixel, (ii) a confidence level that a peak has been correctly identified, (iii) an accuracy estimate for the identified peak, etc. Based on the metric(s) determined for a respective pixel and / or group of pixels, a decision may be made as to whether or not to disable that pixel and / or pixels group, namely, to deactivate respective SPAD(s) of the sensor array. Optionally, such selective disabling and / or deactivation of pixel(s) (i.e. dynamic gating) may then be carried out, e.g., for a remainder of the measurement and / or at least part thereof, in order to save power and / or comply with other specifications and / or requirements of a measurement policy predefined, for example, imposing threshold(s) on determined metric(s) and / or the like.

[0082] In some embodiments, histograms from multiple pixels may be grouped together and data therein may be analyzed collectively. The combined, summed histograms may show peaks that are not identifiable in the individual pixel histograms, due to the small number of photons. This may be used to estimate (i) the peak position (i.e. depth estimate) and associated statistics for this group of pixels, and (ii) how many more laser flashes would be required to yield a strong peak in the individual pixels in the group in isolation. This histogram grouping operation may be performed hierarchically and iteratively, merging larger and larger regions until a peak is obtained. For example, pixels looking at a mid-distance vehicle may show that the object could be resolved at higher resolution with double the number of laser flashes, whereas pixels looking at the sky will never give a measurable peak.

[0083] Optionally, real-time information of photon-event times recorded in a histogram per pixel and / or group of pixels may be evaluated every time the laser flashes and the photon histograms updated as reflections arrive at the SPAD pixel array and trigger photon avalanches therein.

[0084] In some embodiments, depth measurements and / or imaging may be performed over multiple frames, for example, at a rate of about 30 frames per second (fps) and / or the like. Monitoring and analysis of photon histograms per pixel and / or group of pixels for pixel activation / deactivation decision making may be performed from start of a measurement, i.e., as capturing and reconstruction of a new frame begins, and continued as it progresses along during a time interval when a measurement takes place, i.e., for a duration length of 1 / 30 of a second, after which all histogram data may be discarded, and new data for a subsequent frame may be gathered.

[0085] Optionally, selective disabling of individual pixels (or groups of pixels) in terms of when and / or on which particular point(s) in time it takes place, may be controlled in order to save and / or conserve power for a frame as it is being captured.

[0086] Additionally or alternatively, a configured policy may be followed to decide when pixels are disabled, based upon each pixel’s histogram statistics and corresponding metrics determined therefrom, e.g., power consumption, estimated time required before a useful peak is available for this pixel, peak accuracy and confidence levels, etc. In some embodiments, scene analysis may be performed and / or used to identify regions of interest (ROIs) in an image, where a higher confidence and / or accuracy level may be required, and a different policy may be followed in those regions. For example, an object identified as a moving vehicle or pedestrian may require high resolution and distance accuracy, whereas an area identified as the sky or sun does not require an accurate depth measurement.

[0087] Optionally, scene information may be obtained by analyzing an intensity image generated according to an overall amount of light falling onto each pixel, as may be determined from the individual pixel histograms, e.g., by cumulative summation over all bins.

[0088] Optionally, the scene analysis may use data from one or more previous frames, where applicable. The data of an individual preceding frame may include at least one of a depth image and an intensity image (e.g., conventional grayscale image).

[0089] Optionally, the scene analysis may employ a Simultaneous Localization and Mapping (SLAM) algorithm and / or likewise procedure for determining ego-motion and scene reconstruction at once.

[0090] In some embodiments, a per-pixel laser flash count, denoting a number of laser flashes that correspond to times or events in which a respective pixel was activated, may be tracked and stored. Each pixel may be associated with its own dedicated counter, which may be reset to zero at the start of the measurement (e.g., upon each new frame to be captured). Every time the laser flashes (typically as many times as several hundreds or thousands per frame), the timing unit that controls the laser may send a signal to all the flash counters, and counters for enabled pixels may increment the count value by one. The counters for disabled pixels are left unchanged. At the end of the measurement (or respective frame capture duration), the counter value shows the total number of laser flashes during which the associated pixel has been active. To put it another way, this is the duration that the pixel has been active, measured in units of laser flashes.

[0091] Optionally, the per-pixel flash count during pixel activation may be utilized to compute the detection duration estimate (i.e., to estimate how many more flashes are needed for depth detection at desired accuracy and / or confidence). For example, if there have been 100 flashes so far while a pixel has been enabled, and the pixel metrics analysis suggests that 50% more flashes are needed, then one can determine that 50 more flashes are needed for this pixel. The percentage or ratio of further flashes required relative to a total number of flashes emitted thus far may be estimated, e.g., from the hierarchy level and / or group size of pixels block for which individual pixel histograms were merged until a peak can be detected at desired accuracy and / or confidence level, as discussed herein. Additionally or alternatively, scene information from analysis of a current and / or previous frame(s) may be used in assessment of the relation or proportion of additional more flashes required, e.g., based on an accuracy and / or confidence level associated with a region of interest identified, and / or the like.

[0092] Optionally, the pixel activation flash count may be utilized to reconstruct the grayscale picture taken by the sensor, i.e., by adjusting the per-pixel photon-event count in the histograms, as obtained from the measurement, according to the per- pixel flash count recorded. As discussed herein, the histograms as computed by 3D depth sensors such as dToF LiDAR imagers may be converted to a 2D picture, by adding up the total number of photons in each histogram. Ordinarily, the per-pixel photon count determines how bright that pixel would be in the photo (the intensity image). But if the pixel has been deactivated early, its photon count is lower. For example, if a pixel was deactivated after 10% of the laser flashes, the true brightness is 10-fold greater than transpires from the photon count in the histogram alone. To get the true brightness for a pixel, one would have to divide the photon count in the histogram by the laser flash counter value for that pixel, and multiply the result by the total number of laser flashes in the measurement (e.g., per frame), to obtain a correct intensity value for the pixel at hand. Optionally, the 2D photo image can be used for object detection and identification, and selecting regions of interest for a next frame, as discussed herein.

[0093] In addition, the pixel activation flash count may be utilized in other situations and / or applications as well, for example, when correcting histograms to remove the effects of SPAD dead time and / or the like, as known in the art.

[0094] In some embodiments, the laser itself could also be adjusted to save power, using similar pixel metrics analysis as discussed herein. For example, in contrast to scenarios where the only way to save power is to switch off pixels due to usage of an omnidirectional laser that cannot be directed, there may be provided some other laser type(s) that can be focused on particular areas. In those systems, one can use the dynamic gating approach as described herein on both the pixels and the laser. In other words, if it is decided to switch off a pixel, the part of the laser beam that illuminates that pixel could also be switched off concurrently with the pixel.

[0095] One technical effect of utilizing the disclosed subject matter is to provide for advantageous power consumption with regard to relatively close objects. As discussed herein, nearby objects give rise to strong reflections (due to the 1 / R2relationship between intensity and distance, for an object covering multiple pixels). This provides for a strong, clear histogram peak after a small number of laser flashes. Thus, by monitoring and analyzing histogram contents as they are updated throughout the measurement in accordance with the disclosed subject matter, it can be determined that an accurate peak has already been detected after a small number of laser flashes, and the respective pixels can be switched off to save power.

[0096] Another technical effect of utilizing the disclosed subject matter is to for advantageous power consumption in bright illumination conditions. As discussed herein, direct sunlight produces a very high photon flux, which repeatedly triggers the SPADs in the sensor, in the pixels that are focused on the sun. Thus, by utilizing histogram monitoring and analysis according to the disclosed subject matter, it can be determined that the power consumption in those pixels is high, and that they cannot form a detectable histogram peak, even when binned together. Therefore, these SPADs may be disabled to save power.

[0097] Yet another technical effect of utilizing the disclosed subject matter is to provide for advantageous power consumption under diffused scattering (e.g., fog) conditions. A diffuse scattering medium such as fog and / or the like creates strong reflections that continuously excite the SPADs. This would normally lead to high power consumption, because the SPADs would be repeatedly triggered multiple times for each laser flash. However, by utilizing the disclosed subject matter, measurement may be halted early for affected pixels. As discussed herein, the measurement metrics (and optionally grouping / binning) may be used to assess that a depth measurement peak will not appear before the energy consumption thresholds are breached. The affected pixels / SPADs would be switched off, thus saving power.

[0098] Other and / or additional technical problems dealt with, solutions and / or effects of the disclosed subject matter shall become apparent from the discussion herein further.

[0099] It will be appreciated that while, in sole interest of facilitating better understanding and / or ease and convenience of discussion, the disclosed subject matter is described and illustrated herein mainly in context, implementation and / or deployment within dToF LiDAR sensors and / or imaging systems, it is not meant to be limited in such manner, and may be used and / or applied similarly in connection with other likewise ranging sensors, such as may be employed in automotive applications and / or other technological fields, practical scenarios, etc.

[0100] In particular, the disclosed subject matter may be used in any LiDAR sensor with a pixel array. For example, LiDAR based on other methods, e.g. Frequency Modulated Continuous Wave (FMCW) and other likewise techniques. In these sensors, some pixels could be switched off early in each frame to save power. Additionally, in FMCW sensors, for example, there is significant per-pixel computation required to determine the depth peak. This could be avoided for regions that are clearly of lesser interest and / or overly distant, e.g. part of the sky, which could be detected by the previous frame analysis or by the hierarchical pixel grouping.

[0101] Similarly, the disclosed subject matter could also be used also in other types of ranging sensors including for example acoustic sensors such as ultrasound systems and / or the like that are provided with a pixel array and / or a signal transmission source having direction and / or focusing capabilities at a pixel level. For example, in ultrasound systems having a sensor array, the ultrasonic transducer pixels can use a lower duty cycle depending upon the strength of the return signal. Additionally or alternatively, the hierarchical grouping could be used to identify areas that will not yield a useful return and so these areas may be measured less frequently, or computations may be performed with lower resolution in these areas.

[0102] Before explaining at least one embodiment in detail, it is to be understood that embodiments are not necessarily limited in its application to the details of construction and the arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or the Examples. Implementations described herein are capable of other embodiments or of being practiced or carried out in various ways. Embodiments may be a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the embodiments.

[0103] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a readonly memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiberoptic cable), or electrical signals transmitted through a wire.

[0104] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0105] Computer readable program instructions for carrying out operations of embodiments may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of embodiments.

[0106] Aspects of embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0107] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0108] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0109] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardwarebased systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0110] Reference is now made to FIG. 5 which is a schematic illustration of an exemplary use case of a depth sensor in an automobile for parking assistance, according to some embodiments.

[0111] Reference is also made to FIG. 6 which is a schematic illustration of respective exemplary depth map and intensity image produced by a depth sensor, according to some embodiments.

[0112] As shown in FIG. 5, in an exemplary use case scenario, a depth sensor may be used in an automobile for alerting and / or assistive functions, such as for example, as a parking assistance sensor and / or the like.

[0113] The depth sensor may be mounted on the vehicle. One depth sensor may be used, or alternatively multiple depth sensors may be used for a greater coverage, accuracy and reliability. The depth sensor may be a LiDAR sensor, e.g. a dToF LiDAR imaging system. The laser accompanying the LiDAR sensor may emit precisely timed pulses of light, and the LiDAR sensor may receive the reflections from surfaces within its field of view.

[0114] The sensor may measure the distance for every pixel in its field of view. The sensor may also measure the amount of light falling onto each pixel, like a conventional camera. This yields two images, which are perfectly aligned with each other:

[0115] A depth map (i.e. a depth estimate per pixel);

[0116] A grayscale conventional image (an intensity or brightness image).

[0117] FIG. 6 presents on the left side an illustration of a depth image produced by the depth sensor for the scene illustrated on FIG. 5, and on the right side an illustration of the grayscale image produced by the depth sensor for that scene.

[0118] In some embodiments, the scene may be analyzed to decompose this depth map into surfaces, obstacles etc. Object detection and identification may be performed. This information may be used to, for example, do one or more of the following:

[0119] Avoid collisions between the automobile and obstacles;

[0120] Avoid collisions with pedestrians or other road users;

[0121] Assess whether the automobile would fit into the space intended for parking;

[0122] Track the location of the automobile as it navigates into the space for parking; etc.

[0123] The internal structure and design of the sensor in accordance with some embodiments of the disclosed subject matter are discussed in further detail herein.

[0124] Reference is now made to FIG. 7 which is a block diagram of an exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system, according to some embodiments. As shown on FIG. 7, in some embodiments, a Histogram Monitoring Module (or Monitoring Module in short) may be provided for performing one or more functions or acts of sensor array dynamic gating and / or carrying out instructions with relation thereto. The Histogram Monitoring Module may analyze histograms of photon arrival times as they are being collected, and may have the ability to control and configure the dToF LiDAR sensor. The Histogram Monitoring Module may enable and / or disable groups of SPADs according to results obtained for such analysis on the collected histograms.

[0125] The Histogram Monitoring Module may comprise and / or be coupled to a Histogram Data Module (not shown), which may collect raw data of the histograms of photon arrival times and / or derive statistics of their contents, and provide the data to the Histogram Monitoring Module and / or one or more of its subcomponents for further processing, storage, and / or the like. The statistics derived from the histograms’ data may include information such as, for example, a respective peak value of photon events counted at a single bin in each histogram, a respective cumulative sum of counts of photon events over all bins in each histogram, and / or the like.

[0126] The Histogram Monitoring Module may comprise a Pixel Metric Estimator, which may use statistics derived from the histograms (sum, peak value, etc.), as may be obtained from the Histogram Data Module, and may compute metrics including for example: (a) energy consumption metrics for each pixel, (b) confidence level that the histogram peak has been correctly identified, (c) accuracy estimate for the identified peak, and / or the like.

[0127] The Pixel Metric Estimator may compute the energy consumption for a pixel from the histogram’s sum, as it denotes a total number of photon events (i.e., avalanches) at the pixel’s SPAD. The confidence level may be computed by the Pixel Metric Estimator from the Signal to Background Ratio (SBR), obtained for example by a comparison between the peak value (height / number of events) and the level of the underlying background noise, denoted by height (average, median, maximal, and / or the like) of event counts in the remainder of bins of the histogram. The accuracy estimate may be determined by the Pixel Metric Estimator according to known parameters and / or characteristics of the sensor and its physical composition, that may allow accordingly to predict an accuracy level under specified conditions. Such properties may comprise, e.g., factors and / or mechanisms that may contribute to distortions in the histogram (i.e., peak shifting, shot noise, etc.), such as for example, a duration of a dead time after a photon striking in which an affected SPAD in the pixel array need to be recharged, a shape and / or precision of the laser pulse (e.g., whether it is short and rapidly reaches its peak, or more spread out in time, ramps up more slowly and / or has after pulses, etc.), and / or the like. When examining contents of the histogram, such as for example, strength of a peak and / or its position (i.e., distance travelled by the laser flash reflection from the object back to the sensor), the SBR, spread and / or level of background noise, and / or the like, a judgement may be made as to how accurate the measurement is, given the known performance benchmarks of the sensor. For example, for objects at some specified distance away, the depth measurement may be accurate down to a resolution of 1 cm, whereas for farther away objects and / or under different conditions it may be of only 10 cm or even much coarser.

[0128] The Histogram Monitoring Module may comprise a Proceed / Abort Decisions Module (or Proceed / Abort Module in short), which may use the pixel metrics (peak confidence and / or accuracy, energy consumption, and / or the like) from the Pixel Metric Estimator to determine whether to proceed collecting photons in each pixel (SPADs remain enabled), or whether to halt photon collection (SPAD is disabled). In some embodiments, the Proceed / Abort Module may make such determination in accordance with a configured policy for accuracy requirements, peak confidence and / or power consumption. Optionally, these requirements may differ for different parts of the scene, using scene information as may be obtained by analysis of an intensity image generated from the histograms’ sums, as discussed herein.

[0129] In some embodiments, the Histogram Monitoring Module may comprise a data store of per-pixel laser flash count, for recording and / or updating a number of flashes emitted by the laser source for which a SPAD pixel has been active or enabled. The Histogram Monitoring Module may obtain a total number of laser flashes emitted throughout the measurement, as denoted by a laser flash counter, which when taken together with the stored per-pixel laser flash count, may be used by the Proceed / Abort Module for adjusting a pixel brightness value (i.e., histogram sum) in the intensity image, e.g., as a preprocessing step prior to performing analysis thereof for obtaining scene information. Additionally or alternatively, the Proceed / Abort Module may use the stored per-pixel laser flash count and / or the laser flash counter to compensate for histogram distortions such as due to SPAD dead time and / or the like, and check compliance with the configured policy of the modified pixel accuracy and / or confidence metrics, for example.

[0130] Reference is now made to FIG. 8 which is a block diagram of another exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system using pixels block data analysis, according to some embodiments.

[0131] As shown in FIG. 8, in some embodiments, there may be provided a Histogram Monitoring Module similarly as described and illustrated with reference to FIG. 7 herein, and that may further comprise a Hierarchical Histogram Grouping and Binning Module (or Grouping and Binning Module in short), which may combine the histograms from adjacent pixels. If the pixels have histograms with too few photons contained in each, the peak cannot be identified reliably, due to shot noise with Poisson statistics, as discussed herein. When the histograms of a group of adjacent pixels are summed, the peak of the combined histogram can be more reliably determined. Depending upon the quality of the detected peak, this grouping and summation may have to be performed multiple times at different scales, until a peak reliably emerges. This information may be used to estimate how much longer the pixels need to remain enabled, for a strong peak to be identified at smaller scales (e.g. down to one pixel).

[0132] For example, assuming block merging of pixels and / or pixels groups had to be performed 5 times to provide a combined histogram with peak identification at sufficient confidence level, meaning that the merged region doubled 5 times, namely, a group of 25= 32 pixels formed for which all histograms had to be summed together such that both a peak and a background noise level in the combined histogram could be spotted and differentiated from one another, such that the SBR (for the whole group) can be computed and checked for compliance with the configured policy, for example. The fact that there had to be assembled together 32 times as much information in order to get useful histogram contents, thus implies that for each single pixel among that group, one may be required to continue collecting reflections 32 times longer than done so far, so as to collect 32 times more photons, as an initial estimate.

[0133] The Proceed / Abort Module may make determination in view of the estimated exposure multiple for useful peak, as indicated by the hierarchy level of the merges required to be performed at the Grouping and Binning Module, whether or not to continue in collection of photon reflections at SPADs of that group of pixels. For example, the Proceed / Abort Module may check whether spending the additional time required for collecting photons for the pixels group may violate the configured policy, e.g., consume energy above a prescribed quota. As another example, if the group of pixels does not qualify as a region of interest, i.e., based on scene analysis and / or the like, it is classified as belonging to an object that need not be detected at great accuracy (e.g., a tree in the distance, in the automobile parking assistance sensor use case), the Proceed / Abort Module may make determination according to the configured policy whether to continue activation of the SPADs further to achieve more accurate measurement or deactivate them and settle for the inaccurate result.

[0134] In some embodiments, the Proceed / Abort Module may obtain and use the laser flashes count and / or stored per-pixel flash count to compute the detection duration estimate (to estimate how many more flashes may be needed). For example, if there have been 100 flashes so far while a pixel has been enabled, and the estimated exposure multiple for useful peak suggests that 4 times as many flashes are needed, then the Proceed / Abort Module may determine that 400 flashes in total, i.e., additional 300 flashes, are needed for this pixel. The Proceed / Abort Module may then decide accordingly whether or not to disable the pixel in view of the configured policy and how the added number flashes would comply with it or not.

[0135] Reference is now made to FIG. 9 which is a schematic illustration of combining histogram data from multiple pixels of a sensor array, according to some embodiments.

[0136] As illustrated on FIG. 9, grouping together neighboring pixels may allow a previously unseen peak to emerge in the combined (summed) histogram. The size of the grouping area may vary according to the confidence level in the peaks that emerge. This summation of individual pixels’ histograms may be used to identify objects and obstacles that were previously undetectable.

[0137] As shown in FIG. 9, multiple noisy histograms (such as depicted on the left) may be summed together, thus yielding a single, combined histogram (such as depicted on the right), to produce a much clearer peak with improved confidence and accuracy. Reference is now made to FIG. 10 which is a schematic illustration of pixels block grouping in a sensor array, according to some embodiments.

[0138] In the diagram presented on FIG. 10, large shapes represent pixels that have been grouped together to improve confidence. Small shapes (i.e., unit rectangles) represent areas that have not been grouped together (those pixel histograms are sufficiently high quality for a usable peak to be measured without the need to group pixels).

[0139] At each hierarchy level, the confidence metric per pixel and / or pixels group may be assessed. If the confidence level is low, the pixel groups may be merged again to create larger merged regions, until the required confidence level may be reached.

[0140] Reference is now made to FIG. 11 which is a block diagram of yet another exemplary module for dynamic gating of a pixel array in a dToF LiDAR imaging system using previous frame(s) data analysis, according to some embodiments.

[0141] As shown in FIG. 11, in some embodiments, there may be provided a Histogram Monitoring Module similarly as described and illustrated with reference to FIG. 7 herein, and that may further comprise and / or be in communication with a Previous Frame Analysis Module (not shown) for providing data and analysis of one or more previous frames to the Histogram Monitoring Module and / or Proceed / Abort Module thereof.

[0142] The Previous Frame Analysis Module may use a 3D depth map of a previous frame (and / or other image data) to identify features and objects of interest, such as for example, vehicles or obstacles in the exemplary use case of automobilemounted depth sensor for parking assistance, as discussed with reference to FIGS. 5 and 6 herein. The accuracy and correct- peak confidence requirements for these objects may be enhanced, to provide greater certainty of the position and motion of objects that are relevant to the depth sensor’s end use.

[0143] The data of previous frame(s) captured by the sensor (i.e., depth map and / or intensity image) may be analyzed by the Previous Frame Analysis Module to give further insight into the scene. Scene information may also be derived from the current frame as it is being acquired, as discussed herein. This may be used to configure the sensor to require greater / lesser accuracy in certain regions of interest (ROIs). For example, in the automotive scenario, areas of sky may be identified and discarded early (these have no measurable depth, but consume high power). Objects detected as “objects of interest” (e.g. vehicles, pedestrians) may need higher 3D depth accuracy, and therefore these should allow more laser flashes for greater accuracy at corresponding pixels. Areas of the scene with fast motion may similarly have enhanced accuracy requirements. Areas of the scene corresponding to unknown or unidentifiable objects may similarly yet need greater accuracy in order to identify them.

[0144] Moving vehicles typically use a range of techniques to determine their position, and the positions of landmarks around them; one such method is simultaneous localization and mapping (SLAM). Landmarks identified in SLAM may need greater accuracy, to improve the ego-positioning performance and map-building accuracy. Therefore, these landmarks may be indicated as regions-of-interest to the sensor.

[0145] The Proceed / Abort Module may obtain from the Previous Frame Analysis Module identification and / or classification of objects and / or ROIs in the current frame and an indication of accuracy requirements for each, and make determinations regarding activation / deactivation of SPAD pixels accordingly and / or in view of the configured policy, which may vary from one region to another based on the different requirements specified.

[0146] It will be appreciated that the disclosed subject matter provides for real-time determination of per-pixel depthmeasurement performance, and for usage thereof to reduce power consumption for nearby objects or strongly reflective objects.

[0147] It will be further appreciated the disclosed subject matter provides for hierarchical histograms grouping and binning and prediction thereby when (and whether) pixels will successfully produce a depth measurement, thus allowing for power consumption improvement with unmeasurable objects (e.g. sky, sun) that will never yield a peak in the histogram.

[0148] It will be yet further appreciated the disclosed subject matter provides for derivation of per-pixel energy consumption metric for real-time decision-making, thus enabling identification of areas of the scene that consume the most energy in the sensor. It will be yet further appreciated the disclosed subject matter provides for real-time control of SPAD activation (in pixels / small groups) according to performance metrics and ability to switch off areas with excessive power consumption (e.g. block off the sun), where identification and deactivation of affected pixels may be done even at an early stage of a measurement.

[0149] It will be yet further appreciated the disclosed subject matter provides for previous frame(s) analysis to identify differing accuracy requirements for objects within the scene, and usage thereof in power budgeting which may accordingly be more efficiently allocated to regions of interest to the application / scenario.

[0150] It is appreciated that one or more of the acts and / or functions described herein, particularly, but not exclusively, with reference to FIGS. 7-11 herein, may be executed by a computer program for dynamic gating of a pixel array in aranging sensor, the computer program comprising program instructions for executing the aforementioned one or more acts which may be executed by at least one processor. Each of these programs may be provided on a non-transitory storage medium.

[0151] It is further appreciated that one or more of the acts and / or functions for sensor array dynamic gating described herein may be performed, implemented, realized and / or otherwise provided for by a processing circuitry, which may include, utilize and / or otherwise facilitate one or more hardware modules (elements), for example, an electronic circuit, an electric component, an Integrated Circuit (IC), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), an Arithmetic Logic Unit (ALU), a Digital Signals Processor (DSP), a Central Processing Unit (CPU), a Graphic Processing Units (GPU), an Artificial Intelligence (Al) accelerator and / or the like.

[0152] In some embodiments, the processing circuitry may execute one or more software modules such as, for example, a process, a script, an application, an agent, a utility, a tool, an Operating System (OS) and / or the like each comprising a plurality of program instructions stored in a non-transitory medium (program store) coupled to, comprised by and / or otherwise being in communication with the processing circuitry.

[0153] Additionally or alternatively, the processing circuitry may include, utilize and / or otherwise facilitate a memory and / or data storage device which may be provided with a dedicated logic and / or in-memory processing, allowing for producing on its own simple metrics (e.g., total photon count, highest value) of histograms as they are being collected and / or updated therein. For example, in some embodiments, the memory or data store of the histograms may count the photon events going into the histogram to get the sum, and maintain a record of the highest histogram bin value so far.

[0154] Similarly, one or more of the acts and / or functions for sensor array dynamic gating as described herein may be performed, implemented, realized and / or otherwise provided for by a dedicated / hardwired control logic and / or other likewise special purpose hardware, firmware, middleware, and / or the like which may optionally be deployed within the sensor and / or otherwise communicating therewith to control its operation.

[0155] The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0156] It is expected that during the life of a patent maturing from this application many relevant ranging sensors, range imaging systems, and range detection tools and techniques will be developed and the scope of the terms “ranging sensors”, “range imaging systems”, “range detection”, and / or “ranging” is intended to include all such new technologies a priori.

[0157] As used herein the term “about” refers to ± 10 %.

[0158] The terms "comprises", "comprising", "includes", "including", “having” and their conjugates mean "including but not limited to". This term encompasses the terms "consisting of' and "consisting essentially of'.

[0159] The phrase "consisting essentially of' means that the composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not materially alter the basic and novel characteristics of the claimed composition or method. As used herein, the singular form "a", "an" and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0160] The word “exemplary” is used herein to mean “serving as an example, instance or illustration”. Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments.

[0161] The word “optionally” is used herein to mean “is provided in some embodiments and not provided in other embodiments”. Any particular embodiment may include a plurality of “optional” features unless such features conflict.

[0162] Throughout this application, various embodiments may be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of embodiments. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1 , 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0163] Whenever a numerical range is indicated herein, it is meant to include any cited numeral (fractional or integral) within the indicated range. The phrases “ranging / ranges between” a first indicate number and a second indicate number and “ranging / ranges from” a first indicate number “to” a second indicate number are used herein interchangeably and are meant to include the first and second indicated numbers and all the fractional and integral numerals therebetween.

[0164] It is appreciated that certain features of embodiments, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of embodiments, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements.

[0165] Although embodiments have been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0166] It is the intent of the applicant(s) that all publications, patents and patent applications referred to in this specification are to be incorporated in their entirety by reference into the specification, as if each individual publication, patent or patent application was specifically and individually noted when referenced that it is to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting. In addition, any priority documents) of this application is / are hereby incorporated herein by reference in its / their entirety.

Claims

CLAIMS1. A processing circuitry for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the processing circuitry comprising a memory and is configured to: analyze at least one histogram of reflection arrival times counts of at least one pixel of a frame; determine at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disable reflections collection for the at least one pixel according to the at least one metric and a collection policy.

2. The processing circuitry of claim 1 , wherein analysis of the at least one histogram comprises calculating at least one statistic from counts of the at least one histogram.

3. The processing circuitry of claim 2, wherein calculating the at least one statistic comprises calculation of at least one of: at least one peak value of counts of the at least one histogram; and a cumulative sum of values of counts of the at least one histogram.

4. The processing circuitry of claim 2, wherein the at least one metric is computed from the at least one statistic, and wherein the at least one metric comprises at least one of energy consumption, confidence level, accuracy estimate, and detection duration estimate.

5. The processing circuitry of claim 2, further configured to: count for a pixel of the frame a number of transmissions by a signal source of the ranging sensor corresponding to activation timings of the pixel, and compute from the at least one statistic the at least one metric according to the number.

6. The processing circuitry of claim 1, wherein the at least one histogram comprises a plurality of histograms of neighboring pixels.

7. The processing circuitry of claim 1 , wherein the collection policy is varied according to at least one region of interest.

8. The processing circuitry of claim 7, wherein the at least one region of interest is determined through analysis on at least one of a depth map and an intensity image of at least one other frame preceding to the frame.

9. The processing circuitry of claim 7, wherein the at least one region of interest is determined through analysis on scene information of an intensity image generated from histograms of at least one subset of pixels.

10. The processing circuitry of claim 9, wherein the analysis on scene information of the intensity image is performed using a simultaneous localization and mapping, SLAM, process.

11. The processing circuitry of claim 9, further configured to: count for a pixel of the frame a number of transmissions by a signal source of the ranging sensor corresponding to activation timings of the pixel, and adjust a count value obtained as a function of a count in a histogram of the pixel using the number.

12. The processing circuitry of claim 11, wherein the count value is adjusted according to a ratio between the number of transmissions counted for the pixel and a total number of transmissions by the signal source for the frame.

13. The processing circuitry of claim 1, wherein a signal source of the ranging sensor is directional, and wherein the processing circuitry is further configured to selectively disable, according to the at least one metric and the collection policy, a respective beam portion of the signal source at a direction corresponding to the at least one pixel.

14. A method for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the method comprising: analyzing at least one histogram of reflection arrival times counts of at least one pixel of a frame; determining at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disabling reflections collection for the at least one pixel according to the at least one metric and a collection policy.

15. A computer program for dynamic gating of a pixel array in a ranging sensor using signal transmissions and reflections, the computer program comprising program instructions which, when executed by at least one processor, cause the at least one processor to: analyze at least one histogram of reflection arrival times counts of at least one pixel of a frame; determine at least one metric for the at least one pixel based on analysis on the at least histogram; and selectively disable reflection collection for the at least one pixel according to the at least one metric and a collection policy.

Images