Circuitry, electronic device and method for estimating depth

Abstract

The disclosure pertains to a circuitry for estimating a depth. The circuitry is configured to obtain an illumination profile of the illuminator; obtain a histogram from the macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

Description

[0001] CIRCUITRY, ELECTRONIC DEVICE AND METHOD FOR ESTIMATING DEPTH

[0002] TECHNICAL FIELD

[0003] The present disclosure generally pertains to a circuitry, an electronic device and a method for estimating a depth.

[0004] TECHNICAL BACKGROUND

[0005] It is generally known to estimate a depth based on Time-of-Flight imaging. A Time-of-Flight (ToF) camera is a range imaging camera system that determines the distance of objects by measuring the time of flight of a light signal between the camera and the object for each point of the image.

[0006] For example, direct Time-of-Flight (dToF) imaging is based on emitting a light pulse into a scene and receiving a reflection of the light pulse from the scene with a single-photon detector. The single-photon detector counts photons received within each of a plurality of sampling intervals. The reflected light pulse is detected based on a histogram that indicates the photon count per sampling interval, and a position of a rising edge of the reflected light pulse in the histogram corresponds to the depth.

[0007] Although there exist techniques for time-of-flight imaging, it is generally desirable to provide an improved circuitry, system and method for estimating a depth.

[0008] SUMMARY

[0009] According to a first aspect, the present disclosure provides a circuitry for estimating a depth, wherein the circuitry is configured to: obtain an illumination profile of an illuminator; obtain a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predetermined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

[0010] According to a second aspect, the disclosure provides an electronic device for estimating a depth, wherein the electronic device includes: an illuminator; a macropixel that includes a plurality of single-photon detectors; and a circuitry that is configured to: obtain an illumination profile of the illuminator; obtain a histogram from the macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

[0011] According to a third aspect, the disclosure provides a method for estimating a depth, wherein the method includes: obtaining an illumination profile of an illuminator; obtaining a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimating, from the histogram, an active illumination profile; estimating a confidence of the estimated active illumination profile; and estimating a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

[0012] Further aspects are set forth in the dependent claims, the drawings and the following description.

[0013] BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Embodiments are explained by way of example with respect to the accompanying drawings, in which:

[0015] Fig. 1 illustrates active illumination on a sensor for different reflectivity, which includes light intensities over time of active light reflections returned from objects located at the same distance with different reflectivity;

[0016] Fig. 2 shows a plurality of histograms acquired by a SPAD-array dToF imaging apparatus for reflections returned from objects located at the same distance with different reflectivity and corresponding to the active light intensities of Fig. 1.

[0017] Fig. 3 illustrates the range walk delay per active illumination of reflections returned from objects of different reflectivity located at the same distance and corresponding to the active illumination and histograms of Figs. 1 and 2.

[0018] Fig. 4 illustrates estimated active illumination profiles for reflections from objects of different reflectivity located at the same distance corresponding to the histograms of Fig. 2.

[0019] Fig. 5 illustrates the weights, i.e., the estimated confidence, of the estimated active illumination profiles of Fig. 4. Fig. 6 illustrates the normalized weighted - based on the estimated weights of Fig. 5 - correlations between the estimated active illumination profiles of Fig. 4 and sliding illumination profiles of the illuminator for varying offsets.

[0020] Fig. 7 illustrates the interpolation of the normalized correlation deltas over time based on the correlations of Fig. 6.

[0021] Fig. 8 shows the residual range walk delay based on the normalised correlation deltas of Fig. 7 for reflections of different active illumination returned from objects of different reflectivity located at the same distance corresponding to Fig. 1.

[0022] Fig. 9 illustrates the overall intensity of the estimated illumination profiles of Fig. 4 and the actual overall intensity on the sensor.

[0023] Fig. 10 illustrates an electronic device and a circuitry according to an embodiment; and

[0024] Fig. 11 illustrates a method 30 for estimating a depth according to an embodiment.

[0025] DETAILED DESCRIPTION OF EMBODIMENTS

[0026] Before a detailed description of the embodiments under reference of Figs. 4 to 11 (also including reference to Fig. 2) is given, general explanations are made.

[0027] Photon counting direct Time-of-Flight (dToF) imaging

[0028] As mentioned in the outset, direct time-of-flight (dToF) imaging is based on emitting a light pulse into a scene, receiving a reflection of the light pulse from the scene, i.e., from the illuminated objects in the scene, with a single-photon detector and measuring the delay between the emitted light pulse and the detected reflection. The delay between the emitted light pulse and the detected reflection, which is returned from the scene, i.e., from the illuminated target objects of the scene, corresponds to a depth, e.g., a depth of the scene or a depth of the illuminated target objects in the scene. Generally, a ToF camera has an illumination unit (for example, an LED or VCSEL, Vertical -Cavity Surface-Emitting Laser) that illuminates a scene with modulated light. A pixel array in the ToF camera collects the light reflected from the scene and measures phaseshift which provides information on the travelling time of the light (time-of-flight), and hence information on distance.

[0029] The direct ToF (dToF) method uses a photon counting (PC) technique pulse method in which the pulse width of the laser pulse generated by the lidar system can be changed. By reducing the pulse width, reflections are more easily distinguishable and higher resolution is achieved. Photon counting ToF systems like the like the dToF system are recording a photon histogram.

[0030] In detail, the single-photon detector counts photons received within each of a plurality of sampling intervals. The reflected light pulse is detected based on a histogram that indicates the photon count per sampling interval. A depth may be determined based on the position of a rising edge of the reflected light pulse in the histogram, that is, the position of a rising edge, in particular the midpoint of the rising edge, may correspond to the depth.

[0031] For example, in a single-photon avalanche diode (SPAD) array dToF imaging apparatus, e.g., a SPAD-array lidar, the depth may be estimated from an acquired histogram. The SPAD-array dToF imaging apparatus may include a plurality of SPADs arranged in an array. The plurality of SPADs may be grouped to macropixels such that a macropixel includes several SPADs. The micropixel may include N x M SPADs, wherein N and M are different or the same integers. For example, a micropixel may include 3 x 3 SPADs. The SPADs may be grouped together in different arrangements or shapes, for example as a rectangular array or in the form of a cross. The SPAD-array dToF imaging apparatus may emit a light pulse into a scene and control the SPADs to detect photons. Upon receiving a reflection (echo) of the emitted light pulse from the scene, i.e., returned from the objects in the scene, a number of photons detected by the SPADs may increase. Upon detecting a photon, a SPAD assumes a triggered state for a predetermined time interval in which the SPAD cannot detect further photons. Current systems are considered to use single photon avalanche diodes (SPADs) as detectors. For determining a time of detection of the light pulse the number of SPADs of a macropixel that are triggered may be acquired in a histogram. In other words, a percentage of SPADs of a macropixel that are saturated may be acquired in a histogram. That is, the degree of activation of the macropixel corresponding to the number of triggered SPADs may be acquired in a histogram.

[0032] The depth is determined based on the delay between the emitted light pulse and the detection of the returned reflection of the light pulse, the delay occurring due to the time of flight of the photons. For determining the depth, certain features of the histogram, e.g., the midpoint of the rising and / or falling edge, can be used.

[0033] Range walk error

[0034] It has been recognized that depth measurement drift may occur, for example, that LIDARs show depth measurement drift, when an increasing number of photons are returned to the sensor, that is when the reflection (echo) detected by the sensor, e.g., macropixel, includes a high number of photons. That is, the stronger the illumination, the steeper the rise of the histogram and the earlier the midpoint is reached, which leads to depth measurement drift, i.e., the so-called range walk error.

[0035] The accuracy of the time of flight measurement is degraded as the properties of the ToF imaging target (object properties), such as material properties, surface reflection characteristics, target distance, influence the number of photons that are returned, i.e., detected by the sensor, as opposed to the number of photons that are absorbed, scattered, dispersed, or the like, on the way from the illuminator to the dToF sensor. In other words, the accuracy of the photon counting and therefore the depth measurement is reduced, that is, the range accuracy is degraded, which is therefore called the range walk error.

[0036] Thus, LIDAR systematic depth measurement errors, i.e., LIDAR range walk errors, may occur in a way that make bright objects seem closer than dark ones.

[0037] The range walk error and its connection to light intensity is explained further with regard to the following Figs. 1 to 3.

[0038] Fig. 1 illustrates the active illumination on a sensor for different reflectivity, which includes light intensities over time of active light reflections returned from objects located at the same distance with different reflectivity. The horizontal axis (horizontal direction of the figure) of Fig. 1 indicates time, e.g., time intervals, for example, sampling intervals or sampling bins. The time intervals may correspond to predefined subsequent time intervals, for example, time intervals in which the SPADs of a macropixel, for example, of a SPAD-array dToF imaging apparatus, are controlled to detect photons. The vertical axis (vertical direction of the figure) corresponds to the active illumination, i.e., light intensity of a returning reflection (echo) of an emitted light pulse, i.e., active light, for example, emitted by the illuminator of a dToF imaging apparatus, on a sensor, for example, the SPAD of a macropixel of a SPAD-array dToF imaging apparatus (e.g., LIDAR). Fig. 1 shows different light intensities reflected from objects of different reflectivity. The different objects include illuminated target objects of different reflectivity located at the same distance. The reflectivity of target objects ranges from high to medium to medium -low to low to very low reflectivity objects. The varying reflectivity may be caused by a surface including surface patterns with different grey levels functioning as target objects. In this sense high to medium to medium-low to low to very low reflectivity may correspond to respective grey levels of for example 72%, 56%, 24% 12% and 6%. As the surface is located in one plane all the surface patterns are located at the same distance. Thus, the trapezoid shapes illustrated in Fig. 1 may be produced based on such a surface with surface patterns of different grey levels. Alternatively, the same effects of varying intensities can also be caused by the varying distance to reflective surface, with the intensity of returned light decreasing with the square of the distance to the surface, e.g., a surface with the same reflectivity but twice the distance returns a quarter of the illumination.

[0039] This explanation regarding objects of different reflectivity and / or distance also applies in regard to the following figures (e.g., Figs. 2 to 9).

[0040] Fig. 1 illustrates that high reflectivity objects return reflections of high light intensities over time (unbroken line) as opposed to less reflectivity objects, which return reflections of less light intensities (dashed and dotted lines)

[0041] As explained above, alternatively, also distance may impact the light intensity. In that sense, objects located farther away may lead to decreased light intensity as opposed to objects that are located closer.

[0042] In this sense objects, e.g., surface patterns, located at the same distance, but with different reflectivity may correspond to different depths and / or objects of the same reflectivity but located at different distances, because the further away an object is located the less light returns from the object.

[0043] Thus, increasing depth and / or objects located far away lead to reflections of high light intensity relative to less depth and / or objects that are located closer.

[0044] Distance may refer to the measuring distance, e.g., distance from the dTof imaging apparatus, that is, the distance from the illuminator and sensor (e.g., macropixel).

[0045] The active illumination over time of Fig. 1 corresponds to an active illumination profile. That is, an active illumination profile includes the information illustrated in Fig. 1, e.g., the light intensities over time caused by active illumination. Therefore, Fig. 1 illustrates illumination profiles 8 for different surfaces. Therein, the illumination profile 8 (unbroken line) based on a reflection returned from high reflectivity object differs, i.e., includes a higher light intensity over time, a higher overall light intensity and a higher maximum light intensity, from the illumination profiles 8 based on a reflection returned from lower reflectivity objects, i.e., medium to very low reflectivity objects (dashed and dotted lines). Moreover, Fig. 1 indicates the lower the reflectivity, the less light is returned and the lower the active illumination over time for the illumination profiles 8. All illumination profiles 8 indicate the same duration from approximately time bin 7 to time bin 16. Fig. 2 shows a plurality of histograms acquired by a SPAD-array dToF imaging apparatus for detected reflections returned from objects located at the same distance with different reflectivity corresponding to the light intensities of Fig. 1 caused by active light. A horizontal axis of Fig. 2 indicates time, that is, for example, a time interval or sampling bin. The sampling bins correspond to predefined subsequent time intervals in which the SPADs of a macropixel of the SPAD-array dToF imaging apparatus (e.g., LIDAR), which comprises 3x3 SPADs, are controlled to detect photons. A vertical axis of Fig.2 indicates the number of SPADs in the macropixel that are in a triggered state (due to detecting a photon) in the respective time intervals of the respective sampling bins. For example, a vertical value of zero indicates that no SPAD of the macropixel detects a photon in a time interval of a respective sampling bin, and a vertical value of 9 indicates that all SPADs of the macropixel detect a photon in a time interval of a respective sampling bin.

[0046] Fig. 2 shows multiple histograms 1 for reflections returned from different scenes (high to very low reflectivity objects, corresponding to e.g., different depths, and / objects of different distances, see Fig. 1 for explanation). A rising edge of the received reflection in the respective histograms 1 is used to estimate a distance to an object of the scene which reflects the light pulse emitted by the illuminator. Corresponding to the respective histograms 1 objects of different reflectivity, i.e., returning reflections of different light intensities, located at the same distance trigger different number of SPADs at different time intervals. In more detail, reflections from high reflectivity objects with a high active light intensity (see Fig. 1) trigger the maximum number of SPADs quickly (unbroken line). By contrast, reflections from less or low reflectivity objects trigger less SPADs, i.e., trigger less maximum number of SPADs of total SPADs available, and reach saturation of maximum number of SPADS triggered slower (dashed and dotted lines). Therefore, the midpoint of the rising edge, i.e., the moment the histogram has reached to half of its maximum intensity, occurs earlier for reflections from high reflectivity objects relative to reflections from less or low reflectivity objects even though the objects are located at the same distance. For example, Fig. 1 shows that histogram 1 corresponding to the medium reflectivity object (dashed line) rises only marginally slower than histogram 1 corresponding to the high reflectivity object (unbroken line), although the medium reflectivity object histogram 1 reaches the maximum number of triggered SPADs, corresponding to the total number of 9 SPADs available, slower. By contrast, the low reflectivity and very low reflectivity object histograms 1 (dashed and dotted lines) trigger less SPADs overall and rise slower than the rest of the histograms 1 corresponding to comparably higher reflectivity objects (high, medium, medium-low) and much slower than the high reflectivity histogram 1 (unbroken line) in particular.

[0047] As the depth measurement relies on the rising edge midpoint, which should be equal for objects located at the same distance but is instead shown due to differences in histograms 1 of Fig. 2 to be different depending on the reflectivity of the object, which corresponds to the light intensity of the returned active light reflection, a measurement error or measurement drift occurs based on the light intensity returned to the sensor. Moreover, the issue of measurement drift also occurs for objects located at different distances, for example, even for objects of equal reflectivity. That is, similar to Fig. 2, objects that are located closer and therefore return active light reflections of higher light intensity (unbroken line) would have a steeper rising flank, as a higher number of SPADs are triggered quicker, as opposed to reflections from objects that are located further away (dashed and dotted lines) and therefore return reflections of less active light intensity (see explanation of Fig. 1 regarding the impact of object distance and object reflectivity on light intensity).

[0048] Fig. 3 illustrates the range walk delay per active illumination for reflections returned from objects of different reflectivity located at the same distance corresponding to the active illumination and histograms of Figs. 1 and 2. Fig. 3 corresponds to Fig. 2 and Fig. 1 in that the horizontal axis corresponds to the maximum light intensity of the reflected active light pulse returning from objects of different reflectivity located at the same distance (which would show similar results for objects located at different distances (of the same reflectivity)). In the same vein, the vertical axis of Fig. 3 corresponds to the midpoint of the rising flank of the histograms of Fig. 2, i.e., the estimated delay or depth, which depends on the different light intensities of the returning active light reflections. Therefore, Fig. 3 illustrates again that a high range walk delay occurs for returning reflections of less intensity relative to a lower range walk delay occurring for returning reflections of a higher intensities. This difference in delay, from which the depth is estimated, for reflections of different light intensities, e.g., for reflections returning from objects of, for example, varying reflectivity, although the distance of objects is the same, but which occurs also for different depths and / or for similar (in terms of reflectivity) objects of different distances, is called the range walk error.

[0049] It has been recognized that the range walk error may be compensated through characterization of the range walk error as a function of different parameters of the histogram shape (Fig. 2). However, it has also been recognized that the range walk error may only be partially compensated by this technique and the compensation requires a complex multivariable characterization, which depends, among other things, on the illumination intensity, shape, ambient light, sensor performance and sub sampling period offset.

[0050] To avoid these shortcomings, a solution to computing the depth while minimizing the impact of the range walk error may proceed in two steps, by first, estimating, from the histograms, the active illumination profile on the (macro)pixel and its confidence, and, secondly, by correlating the estimated active illumination profile with the known illumination profile of the illuminator (via a confidence weighted correlation) and measuring the delays to the points of the maximum / maxima - confidence weighted - correlation. The "active" illumination profile on the macropixel is the illumination profile caused by the active light, e.g., light that varies over time, such as light emitted by the illuminator, as opposed to ambient light.

[0051] There may be one or more maxima of the correlation. That is, the maximum of the correlation may not be unique, and there may be multiple maxima, e.g., if there are multiple echoes of the light pulse (e.g. when illuminating through a semitransparent curtain).

[0052] The known illumination profile of the illuminator refers to the active illumination profile of the illuminator, which may function as a reference active illumination. For example, a (reference) active illumination is illustrated in Fig. 1, which shows the light intensity over time on the sensor which is caused by the active light emitted by the illuminator, i.e., the illumination profile of the illuminator. When there is only one path between the illuminator and the sensor the reference active illumination and the illumination profile as it comes out of the sensor may be the same but for the attenuation and the delay along the path. When there are multiple paths from the illuminator to the sensor there may be a difference, which may lead to multiple maxima in the correlation, one for each path.

[0053] Consequently, some embodiments of the present disclosure pertain to a circuitry for estimating a depth, wherein the circuitry is configured to: obtain an illumination profile of an illuminator; obtain a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predetermined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator by the weighted confidence of the estimated active illumination profile.

[0054] The circuitry may include an application-specific integrated circuit (ASIC), a field- programmable gate array (FPGA), and / or any suitable kind of programmable microprocessor or integrated circuit and / or any other kind of processor. A functionality of the circuitry may be specified at least in part by a hardware configuration of the circuity and / or at least in part by software stored on or provided to the circuitry, wherein the software may include instructions executed by the circuitry. The circuitry may also include a storage, a memory (RAM, ROM or the like) that, for example, stores (e.g., temporary) data that are generated for estimating the depth. The memory may include a flipflop, a latch, a static random-access memory (SRAM), an embedded dynamic random-access memory (eDRAM) or the like. The circuitry may further include a communication unit for receiving data (e.g., the histogram or data necessary for generating the histogram), and / or for outputting data (e.g., the estimated depth). The communication unit may include a processor pin, a Mobile Industry Processor Interface (MIPI) camera serial interface (CSI), a peripheral component interconnect (PCI) interface, a universal serial bus (USB) interface, or the like. Thus, the circuitry may include input means (mouse, keyboard, camera, etc.) and / or output means (display (e.g. liquid crystal, (organic) light emitting diode, etc.), loudspeakers, etc., a (wireless) interface, etc., for example, an interface as it is generally known for electronic devices (computers, smartphones, etc.). Moreover, it may include sensors for sensing still image or video image data (image sensor, camera sensor, video sensor, etc.), for sensing a fingerprint, for sensing environmental parameters (e.g., radar, humidity, light, temperature), etc.

[0055] The circuitry may be configured to perform and / or control direct time-of-flight (dToF) measurements. For example, the circuitry may be configured to emit light pulses into a scene.

[0056] The illumination may be light pulses. The light pulses may be generated by any suitable illuminator (i.e., light emission unit), e.g., by a laser diode, a vertical-cavity surface-emitting laser (VCSEL), a light-emitting diode (LED) or the like. The light pulses may include laser pulses. The light pulses may have any suitable shape. For example, the light pulses may include one or more dots (e.g., an array of laser dots), one or more lines, and / or a diverging light beam that is configured to illuminate the scene in a predetermined solid angle.

[0057] The illumination emitted by the illuminator into the scene may be reflected by an object in the scene. The reflected illumination from the object may be detected by a dToF sensor. The dToF sensor may include one or more macropixels (e.g., a one-dimensional or two-dimensional array of macropixels). Each macropixel may include a plurality of single-photon detectors, e.g., singlephoton avalanche diodes (SPADs), that are configured to detect single photons. The singlephoton detectors of a macropixel may be arranged in a one-dimensional or two-dimensional array. For example, a macropixel may include four single-photon detectors arranged in two rows and two columns, nine single-photon detectors arranged in three rows and three columns, sixteen single-photon detectors arranged in four rows and four columns, or the like. The disclosure is not limited to these arrangements, and the single-photon detectors may be arranged in any suitable number of rows and columns. Some single-photon detectors may be shared among macropixels, e.g., a macropixel may overlap an adjacent macropixel, and photons detected by a certain singlephoton detector of the macropixel (e.g., a single-photon detector in a portion where the macropixel and the adjacent macropixel overlap) may be associated with both the macropixel and the adjacent macropixel.

[0058] Upon detecting a photon, the respective single-photon detector (e.g., SPAD) may assume a triggered state for a predetermined time interval. An activation or saturation of a macropixel may correspond to a percentage (fraction) of all single-photon detectors of the macropixel that are in a triggered state, that is, it may correspond to the number of single-photon detectors that are in a triggered state. For each reflected light pulse detected by the macropixel, degrees of activation or saturation of the macropixel in subsequent time intervals, i.e., predetermined time intervals, may be represented in a respective histogram (see, for example histogram 1 of Fig. 2). The histogram may include predefined sampling bins that correspond to the subsequent time intervals, i.e., the predetermined time intervals, and each sampling bin or each predetermined time interval may be associated with a degree of activation of the macropixel in the respective time interval. A value of zero of the degree of activation may indicate that no single-photon detector of the respective macropixel has detected a photon during a time interval of a respective sampling bin, a value of one of the degree of activation may indicate that one single-photon detector of the macropixel has detected a photon in a time interval of a respective sampling bin and a value of nine of the degree of activation may indicate that nine single-photon detectors of the macropixel, which may be the maximum number of single-photon detectors in the macropixel, have detected a photon in a time interval of a respective sampling bin.

[0059] All sampling bins, i.e., all predetermined time intervals, of the histogram may have a same sampling bin width. The sampling bins of the histogram may have a width of (i.e., may correspond to a time interval of) 0.1 ns, 0.5 ns, 1 ns, 2 ns, 5 ns or 10 ns. However, the disclosure is not limited to these values, and the sampling bins may have other suitable values. The histograms 1 of Fig. 2 are examples of respective histograms in which degrees of activation of a macropixel in subsequent time intervals are represented. An acquisition of the histogram may include controlling the single-photon detectors of the macropixel to detect photons, and acquiring the degree of activation of the macropixel at each respective sampling bin.

[0060] For reducing the range walk error when estimating depth, the illuminator may emit a light pulses into a scene, the dToF sensor (e.g., a macropixel of the dToF sensor) may detect the reflection of the respective light pulse from an object in the scene, and a histogram may be generated that may correspond to the respective light pulse emitted into the scene (and whose reflection from the object may be detected by the macropixel).

[0061] This may be repeated for different scenes, that is, for objects at different distance to the illuminator and / or dToF sensor or for objects of different reflectivity but located at the same distance, or for a combination of both. Thus, reflections of the respective light pulse may vary depending on the object property, e.g., the object reflectivity or object distance, as the light intensities of the active light reflections may vary. Active light refers to the reflection being based on the illumination of the illuminator, which may vary over time, as opposed to ambient light. In this sense, an offset between emitting the light pulse by the illuminator and starting an acquisition of the respective histogram may be varied between the objects of different distance as the time of flight is longer for objects that are further away.

[0062] Thus, acquisition of the histogram may include acquiring a plurality of histograms. Each of the plurality of histograms may be based on reflections of different active light intensities as described in Fig. 1, that is, for example reflections returned from objects of different distance or different reflectivity or a combination of both.

[0063] In other words, each of the plurality of histograms may be associated with a respective light intensity of an active light reflection returned from objects of different properties. That is, an edge (e.g., start or end) of a sampling bin in a first histogram of the plurality of histograms may differ, and the difference may be based on the respective light intensity of the respective active light reflections, which may be based on the respective object property of the scene the emitted light pulse is illuminating (wherein the emitted light pulse property may be equal for all scenes and therefore may not influence the difference in histograms), from a corresponding edge of a corresponding sampling bin in a second histogram of the plurality of histograms.

[0064] The circuitry may obtain the histogram (e.g., one or more histograms). For example, the circuitry may receive the histogram from the dToF sensor or from a processing unit thereof. For example, the circuitry may receive, from the dToF sensor, indications of single photons detected by the macropixel or by the respective single-photon detectors of the macropixel, and may generate the histogram based on the received indications.

[0065] The circuitry may perform processing of the histogram and may estimate the depth based on the processing.

[0066] The depth may correspond to a distance of the object, e.g., a distance between the object and the dToF sensor, or a distance between the object and any suitable reference point or reference plane whose position relative to the dToF sensor is predetermined. Thus, the aforementioned different distance of the objects of different scenes may refer to the distance between the object and the dToF sensor, or a distance between the object and any suitable reference point or reference plane whose position relative to the dToF sensor is predetermined. Naturally, the object of different scenes may be the same placed at different distances.

[0067] The circuitry may obtain the illumination profile of the illuminator. For example, the circuitry may receive the illumination profile from the illuminator or from the dToF sensor or from a processing unit of the illuminator or the dToF sensor.

[0068] The illumination profile of the illuminator may function as a reference illumination profile.

[0069] The illumination profile of the illuminator may be predetermined, for example by the manufacturer. However, due to the range walk error the active illumination profile caused by the illumination of the illuminator may differ from the active illumination profile arriving on the dToF sensor, i.e., the active illumination profile on the micropixel (see Fig. 1). That is, depending on the distance or other property of the object, illumination, i.e., due to the emitted light pulse, may be “lost” on the way to the dToF sensor. The further away the object is located or the less reflective the object is, the more illumination is “lost”, i.e., absorbed, scattered or dispersed, on the way to the dToF sensor. Thus, the obtained illumination profile of the illuminator, which should correspond to the actual detected illumination profile and should be the same for every scene, may in fact differ and may instead vary depending on the illuminated scene. For example, in Fig. 1 it was illustrated that the active illumination differed based on the different scenes of Fig. 1.

[0070] Therefore, the histogram (e.g., histogram 1 of Fig. 3) may be associated not only with the illumination profile of the illuminator, but also with the actual intensity of the reflected illumination (e.g., reflection / echo of the emitted light pulse) and therefore the property of the object returning the reflected illumination. Furthermore, the (reference) illumination profile may be different for each macropixel. That is, the illumination profile of the illuminator may not be unique, and may be measured for each direction in which the illuminator illuminates, allowing e.g. for the compensation of the characteristics of different VCSEL in the illuminator.

[0071] For example, there may be variation in illumination profile coming out of the illuminator as the illuminator may include an array of VCSEL. The different VCSEL of an array may illuminate in different directions. Thus, different directions may be illuminated by different VCSEL. Additionally, also within a VCSEL there may be differences between the different modes of the VCSEL. Thus, there may be variation of the illumination profile per VCSEL, and in turn also per macropixel.

[0072] The circuitry may therefore obtain an illumination profile for each direction in which the illuminator illuminates (e.g., and the sensor senses), for example for each VCSEL. In turn, also the obtained histograms as well as the estimated active illumination profile, the estimated confidence and the estimated depth may accordingly take into account the variation in illumination profile per direction and may therefore be processed for each direction in which the illuminator illuminates, for example for each VCSEL (and in turn also for each micropixel)

[0073] The illumination profile, i.e., the obtained illumination profile of the illuminator, and / or the estimated active illumination profile, may include an illumination intensity, i.e., illumination value, per time. Thus, an illumination profile duration may, for example, correspond to a predefined maximum time of the illumination profile, or it may correspond to the maximum time during which an illumination intensity of the profile is above zero, for example in Fig. 1 the duration of each of the light intensity signals (8, of Fig. 1) may be determined to be 9, as they started to be above zero intensity at time 7 and ended at time 16. Alternatively, the duration may be predetermined, for example, concerning Fig. 1, the duration could be 16, as it may start at time 5 and end at time 21 corresponding to the illustrated lines of the graphs of Fig. 1. The illumination profile may include one or more light intensity signals. Each light intensity signal may include a rising edge and a tail. Part of the tail may be removed from the illumination profile in an optimization process, as the tail may degrade precision in further processing steps. The illumination profile may include a reference illumination, which is a light intensity signal whose value is 0. For efficiency's sake the reference illumination whose value is 0 may be dropped from the illumination profile. Concerning the estimated active illumination profile, time may correspond to subsequent time intervals as described above. That is, the illumination profile may be estimated by estimating the illumination intensity for subsequent predetermined time intervals (i.e., bins).

[0074] The circuitry may perform processing of the histogram for estimating the active illumination profile, that is, estimating the active illumination arriving on the macropixel which may differ from the illumination profile of the illuminator. Thus, depending on the histogram different active illumination profiles may be estimated. Therefore, if processing is performed on multiple histograms, also multiple active illumination profiles may be estimated. Although, the multiple estimated illumination profiles may also be equal to each other. That is the same active illumination profile may be estimated based on multiple histograms. Furthermore, the circuitry may perform processing of the histogram which may include determining or calculating, the confidence of the estimated active illumination profile. For that purpose, the probability that the estimated active illumination profile is correct, e.g., the probability that the estimated active illumination profile is the actual illumination profile arriving on the macropixel, may be determined by the circuitry, which may be based on the estimated active illumination profile, the obtained illumination profile of the illuminator, i.e., the active illumination profile of the illuminator, and / or the histogram. In case of multiple histograms (e.g., the plurality of histograms), wherein multiple active illumination profiles may be estimated, also multiple confidences may be determined, which may differ or may be equal to each other.

[0075] The circuitry may perform processing of the histogram, the estimated active illumination profile, the obtained illumination profile and / or the confidence for correlating the estimated active illumination profile with the obtained illumination profile of the illuminator based on the confidence, and / or estimating the depth, based on the correlation. That is, the circuitry may perform a confidence (e.g., confidence of the estimated active illumination profile) weighted correlation, between the estimated active illumination profile and the obtained illumination profile of the illuminator (i.e., the active illumination profile). Furthermore, the maximum of the correlation may correspond to the depth. Thus, the depth may be estimated by the circuitry based on the maximum of the confidence weighted correlation.

[0076] Estimating the confidence of the estimated active illumination profile may include estimating the confidence for each point of the estimated active illumination profile.

[0077] Estimating a depth may be based on estimating a depth based on the maximization of the correlation of the estimated active illumination profile with the obtained illumination profile of the illuminator, weighted by the confidence of the estimated active illumination profile. In some embodiments, the estimating of the active illumination profile may be based on the difference between consecutive values of the obtained histogram. The consecutive values of the histogram may refer to histogram values of adjacent bins of the histogram. The estimation of the active illumination profile may be based on the difference of consecutive histogram values, because this number increases with illumination intensity.

[0078] In some embodiments, estimating the active illumination profile may be based on the proportion of activation of the macropixel at the start and end time of a predetermined time interval. That is, the estimation of the active illumination profile may be based on the proportion of activation of the macropixel at the start of a time interval, e.g., a bin, and based on the proportion of activation of the macropixel at the end of the time interval, e.g., the end of the same bin as used for the proportion of activation of the macropixel at the start of the bin.

[0079] However, a more precise active illumination profile estimator may take into account that fewer active SPADs can be triggered as more SPADs get triggered. That is because of the so-called dead time, which occurs when a SPAD is detecting a photon and it becomes unable to detect another photon during an amount of time called the dead time.

[0080] Therefore, in some embodiments, estimating the active illumination profile may be based on the Napierian logarithm of a ratio of a proportion of the activated macropixel at the start of a predetermined time interval to a proportion of the activated macropixel at the end of a predetermined time interval multiplied by the maximum value of the histogram. This takes into account the dead time and that fewer SPADS can be triggered as more SPADS get triggered. In other words, the estimation of the active illumination profile may be based on the following: where Inis the intensity of the estimated active illumination profile in the [n — l,n] interval,

[0081] Hmaxis the maximum value of the histogram,

[0082] Pnis the proportion of active SPADs in the histogram at time n i.e., (Hmax— Hn) / Hn, and

[0083] Hnis the value of the histogram at time n.

[0084] In some embodiments, estimating the active illumination profile may be based on the proportion of the macropixel that is reactivated during a time interval. Thus, an even more precise estimation may be achieved by taking into account the proportion of SPADs of a macropixel that is reactivated during the sampling interval, which is proportional to the ambient light level in the histogram until the first SPAD is activated or triggered. For example, this may be based on the following or an approximation of the following: where l'nis the active illumination profile intensity estimate, i.e., the intensity of the estimated active illumination profile, in the [n — 1, n] interval, ambient light included, and

[0085] Ramb is the proportion of SPADs activated by ambient light that recover in one interval, which may be essentially or approximately: amb amb / ^ma>: • Tree , where Hamb is the level of the histogram caused by ambient light, and Trecis the average SPAD recovery time.

[0086] A recovering SPAD may refer to a SPAD that has been triggered and becomes unable to detect another photon during the dead time recovering so that it can be triggered again, i.e., can again detect another photon. That is, a recovering SPAD is a SPAD recovering after the dead time.

[0087] Furthermore, the sampling interval and / or the interval may refer to the aforementioned time interval or predetermined time interval.

[0088] In some embodiments, estimating the confidence of the estimated active illumination profile may be based the histogram values for each time interval. The confidence may also be referred to as weight. The confidence - or weight - of the estimated active illumination profile is high when the illumination value, i.e., illumination intensity, has been estimated with a high confidence and it is low (or even zero) when the illumination value has been estimated with a low (or zero) confidence.

[0089] The confidence value may for instance be proportional to the inverse of the variance of the illumination estimate, which can be estimated by e.g. the geometric mean of two consecutive active pad proportions used to estimate the active illumination profile, or the double of the ratio of their product by their sums i.e.:

[0090] Wn~ Pn-P n-l / (Pn+Pn-1) > where Wnand W'nare two confidence / weight estimators for the [n — 1, n] interval.

[0091] Thus, estimating the confidence of the estimated active illumination profile may be based on: wherein Wnis the confidence of the estimated illumination profile for a predetermined time interval [n-l,n], wherein Pnis the proportion of the activated macropixel at the end of the time interval [n-l,n], and wherein Pn-i is the proportion of the activated macropixel at the start of the time interval [n-l,n].

[0092] In some embodiments, correlating the estimated active illumination profile with the obtained illumination profile of the illuminator may be based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile may be based on the obtained illumination profile of the illuminator. That is, the correlation between the estimated active illumination profile with the obtained illumination profile of the illuminator may be a normalised, weighted, correlation, wherein the weight refers to the confidence of the estimated illumination profile.

[0093] Thus, in a first phase the active illumination profile and confidence may be estimated by the circuitry and in a second phase, a normalised, weighted, correlation may be computed by the circuitry, for example, according to the formula shown hereafter, between the active illumination profile and a sliding illumination profile of the illuminator for varying offsets: where Cnis the normalised weighted correlation for an offset of n,

[0094] Wn+i is the estimated confidence, for the [n+i-l,n+i] interval,

[0095] In+i is the illumination profile intensity estimate, i.e., the estimated active illumination profile, in the [n+i-l,n+i] interval,

[0096] Li is the active illumination profile intensity average, i.e., the average of the intensity of the obtained illumination profile of the illuminator, in the interval imax is the duration of the active illumination profile, i.e., the duration of the obtained illumination profile of the illuminator.

[0097] Thus, estimating the depth may be based on a sliding filter and the sliding filter may be based on: wherein Cnis the normalised weighted correlation for an offset of n, wherein Wn+iis the confidence of the estimated illumination profile for the [n+i-1, n+i] time interval, wherein In+iis the estimated illumination profile intensity in the [n+i-1, n+i] time interval, wherein L, is the obtained illumination profile intensity average in the [i-1, i] time interval, and wherein imaxis the duration of the obtained illumination profile of the illuminator.

[0098] This results in a correlation Cn, that is, it results in multiple correlations Cn(or multiple correlation values Cn), for example, depending on the offset of n, with a maximum at the estimated delay, i.e., corresponding to the estimated depth, which may be further refined, e.g., by interpolating the derivative of the correlation before and after the maximum and computing its intersection with the x- axis indicating zero correlation. In other words, in case of multiple correlations Cnthe maximum correlation Cnof the group of multiple correlations Cncorresponds to the estimated depth. Furthermore, it leads to an estimated depth with a smaller range walk error than estimating the depth based on the rising flank of the histogram as explained above (see Fig. 3).

[0099] Thus, estimating the depth may be based on Cnwith the offset of n for which Cnis maximised.

[0100] In other words, estimating the depth may be based on the normalized weighted correlation for which the normalized weighted correlation is maximised

[0101] In some embodiments, estimating the depth may be based on interpolating the derivative of the correlation before and after the maximum of the correlation. That is, the correlation may include multiple correlation values Cn, i.e., multiple correlations, based on n. Thus, the depth may be estimated by interpolating the derivative of the correlation values before and after the maximum of the correlation values.

[0102] Furthermore, a more precise estimate of the overall intensity of the active illumination profile on the sensor may be obtained. For example based on the following: where A is the estimate of the overall active illumination intensity on the sensor, i.e., the overall intensity of the estimated active illumination profile, n is the offset for which Cnis maximised, and

[0103] Hmax is the maximum value of the histogram, i.e., the acquired histogram from the macropixel. Thus, an overall intensity of the estimated active illumination profile may be estimated based on: wherein A is the overall intensity of the estimated active illumination profile, n is the offset for which the confidence weighted correlation is maximised, and Hmax is the maximum value of the histogram.

[0104] Some embodiments pertain to an electronic device for estimating a depth, wherein the electronic device comprises: an illuminator; a macropixel that includes a plurality of single-photon detectors; and a circuitry which is configured to: obtain an illumination profile of the illuminator; obtain a histogram from the macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator based on the confidence of the estimated active illumination profile.

[0105] It is noted that the aspects described above for the circuitry may be combined in any suitable way, and that the circuitry described above may further exhibit any suitable feature described below with respect to any one of Fig. 4 to 11.

[0106] Some embodiments pertain to an electronic device for estimating a depth, wherein the electronic device comprises: an illuminator; a macropixel that includes a plurality of single-photon detectors; and the circuitry according to any configuration described above.

[0107] For example, also in regard to the circuitry of the electronic device estimating the active illumination profile may be based on the difference between consecutive values of the histogram.

[0108] For example, in some embodiments, the circuitry of the electronic device may be further configured to estimate the confidence of the estimated active illumination profile based on the histogram values.

[0109] For example, in some embodiments, the circuitry of the electronic device may be further configured to correlate the estimated active illumination profile with the obtained illumination profile of the illuminator based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile may be based on the obtained illumination profile of the illuminator. Some embodiments pertain to a method for estimating a depth, wherein the method includes: obtaining an illumination profile of an illuminator; obtaining a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimating, from the histogram, an active illumination profile; estimating a confidence of the estimated active illumination profile; and estimating a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

[0110] The method may be configured corresponding to the circuitry described above, and the method may accordingly exhibit any feature described above with respect to the circuitry and / or any suitable feature described below with respect to any one of Fig. 4 to 11. The method may be performed by the circuitry and / or by the electronic device described above.

[0111] For example, concerning the method, correlating the estimated active illumination profile with the obtained illumination profile of the illuminator may be based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile may be based on the obtained illumination profile of the illuminator.

[0112] The methods as described herein may also be implemented in some embodiments as a computer program causing a computer and / or a processor to perform the method, when being carried out on the computer and / or processor. In some embodiments, also a non-transitory computer- readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.

[0113] Returning to Fig. 4, estimated active illumination profiles are illustrated for reflections from objects of different reflectivity located at the same distance corresponding to the histograms of Fig. 2. Fig. 4 corresponds to the histograms 1 of Fig. 2. That is, the active illumination estimates over time, i.e., the estimated active illumination profiles 2, as shown in Fig. 4 are based on the histograms 1 as illustrated in Fig. 2. As explained above the histograms 1 of Fig. 2 are based on reflections returned from illuminated target objects with different properties. The different properties are different surface reflectivity of the objects that are located at the same distance, which leads to different light intensities returned from the objects and detected by the sensor, e.g., the macropixel, as illustrated in Fig. 1. However, the same effect of different detected light intensities would be achieved by objects located at different distances, as explained with regard to Figs. 1 and 2. Thus, also Fig. 4 is an example of different estimated active illumination profiles 2 for objects located at different distances. The horizontal axis of Fig. 4 indicates time, for example, time intervals, for example, sampling intervals or sampling bins. The time intervals may correspond to predefined subsequent time intervals, for example, time intervals in which the SPADs of a macropixel, for example, of a SPAD-array dToF imaging apparatus, would be controlled to detect photons if the light intensity was detected instead of estimated. The vertical axis corresponds to the active illumination estimate, i.e., the active light intensity estimate, of the estimated active illumination profile. Fig. 4 shows that the estimated active illumination based on objects of high reflectivity, i.e., or for objects located farther away, (unbroken line) is higher than the estimated active illumination based on objects of less or lower reflectivity (dashed and dotted lines). This corresponds to the detected illumination values, for example, as detected by a sensor, such as illustrated and explained in regard to Fig. 1. In other words, the estimated active illumination profiles 2 of the respective object reflectivity correspond to the detected illumination profiles 8 of the respective object reflectivity. That is, estimated illumination intensity of active illumination profiles 2 is higher the higher the reflectivity of the object and lower the lower the reflectivity of the object, which corresponds to Fig. 1, wherein detected illumination intensity is higher the higher the reflectivity and lower the lower the reflectivity. Although some differences may be apparent, for example, between the estimated illumination profile 2 of the high reflectivity object (unbroken line, Fig. 4) to its corresponding detected illumination profile 8 of high reflectivity (unbroken line, Fig. 1), the relative maximum light intensities, compared to the light intensities of objects of less reflectivity (dashed and dotted lines), in particular, between times 7 and 12 are similar, as well as the overall shape of the illumination profiles.

[0114] Fig. 5 illustrates the weights, i.e., the estimated confidence, of the estimated active illumination profiles of Fig. 4. Fig. 5 corresponds to the histograms 1 of Fig. 1. That is, the weight, i.e., the respective confidences 3 of the estimated active illumination profiles 2 of Fig. 4, are based on the respective histograms 1 of Fig. 2. As explained above the histograms 1 of Fig. 2 are based on reflections returned from illuminated target objects with different properties. The different properties are different surface reflectivity of the objects that are located at the same distance, which leads to different light intensities returned from the objects and detected by the sensor, e.g., the macropixel, as illustrated in Fig. 1. However, the same effect of different detected light intensities would be achieved by objects located at different distances, as explained with regard to Figs. 1 and 2. Thus, also Fig. 5 is an example of estimated confidences 3 for different estimated active illumination profiles 2 for objects located at different distances. The horizontal axis of Fig. 5 indicates time, for example, time intervals, for example, sampling intervals or sampling bins. The time intervals may correspond to predefined subsequent time intervals as explained in regard to Fig. 4. The vertical axis corresponds to the estimated confidence i.e., the weight, of the estimated active illumination profiles 2 of Fig. 4. Fig. 5 shows a high confidence at the beginning and a rapidly declining confidence over time for all estimated active illumination profiles 2, wherein the decline is quicker for estimated illumination profiles 2 of early high active illumination, i.e., estimated illumination profiles 2 based on reflections returned from objects with high reflectivity at the same location, i.e., which corresponds to close objects (of the same reflectivity). Thus, confidence 3 is higher for estimated active illumination profiles 2 based on reflections returned from less reflective objects (i.e., which return less light intensity), i.e., which corresponds to objects that are farther away (dashed and dotted lines). In other words, the higher the reflectivity the higher the estimated light intensities of the estimated illumination profiles 2 (Fig. 4) and the lower the confidence 3 over time.

[0115] Fig. 6 illustrates the normalized weighted - based on the estimated weights of Fig. 5 - correlations between the estimated active illumination profiles of Fig. 4 and sliding illumination profiles of the illuminator for varying offsets. The horizontal axis of Fig. 6 indicates time, for example, time intervals, for example, sampling intervals or sampling bins. The time intervals may correspond to predefined subsequent time intervals as explained in regard to Fig. 4. The vertical axis corresponds to the normalized weighted (based on the weights of Fig. 5) correlation between the estimated active illumination profile of Fig. 4 and an active illumination profile of the illuminator that sent an emitted light pulse into different scenes for objects of different reflectivity located at the same distance, which results in the histograms 1 of Fig. 2 on which basis the active illumination profiles 2 of Fig. 4 and the weights 3 of Fig. 5 are estimated. Furthermore, the normalized weighted correlation may be conducted between the estimated active illumination and a sliding illumination profile of the illuminator for varying offsets, i.e., the sliding illumination profile may be based on the expected illumination profile of the illuminator, that is, the correlations of Fig. 6 may be based on: where Cnis the normalised weighted correlation for an offset of n,

[0116] Wn+i is the confidence / weight estimator, i.e., the estimated confidence (3 of Fig. 5) , for the [n+i- l,n+i] interval, In+i is the illumination profile intensity estimate, i.e., the estimated active illumination profile (2 of Fig. 4), in the [n+i-l,n+i] interval,

[0117] Li is the active illumination profile intensity average, i.e., the average of the intensity of the obtained illumination profile of the illuminator, in the interval imax is the duration of the active illumination profile, i.e., the duration of the obtained illumination profile of the illuminator.

[0118] That is for each scene (of different object properties, i.e., different object reflectivity or different object distances) which results in a separate histogram (1, Fig. 2) and therefore results in separate estimated active illumination profiles (2, Fig. 4) and weights (3, Fig. 5) the correlations 4 are illustrated in Fig. 6.

[0119] Fig. 6 shows a maximum 5 for every correlation 4 corresponding to each different scene. The maxima 5 seem to all lie around time 8, which may correspond to the same time interval or the same time bin. The respective maxima 5 of the correlations 4 correspond to the estimated delay, i.e., the delay between emitted light pulse and detected reflection (echo) of the emitted light pulse returned from the respective illuminated scenes and detected by a sensor, for example a dToF sensor, such as a macropixel including SPADs (see Fig. 2 explanation for more details). In other words, the respective maximum correlations (maxima 5) correspond to the estimated depth.

[0120] Fig. 7 illustrates the interpolation of the normalized correlation deltas over time based on the correlations of Fig. 6. The maximum correlation values of Fig. 6 are further refined by interpolating the derivative of the respective correlation 4 before and after the respective maximum 5 as shown in Fig. 7 and computing its intersection 7 with the x-axis of Fig. 7. The x- axis of Fig. 7 illustrates the time interval between time values 7 and 8, i.e., the time before and after the maximum correlation 5 as shown in Fig. 6 for the different correlations 4. That is, the x- axis corresponds to the normalized correlation delta of zero. The y-axis illustrates the normalized correlation deltas, i.e., the interpolated derivative of the correlation before and after the maximum correlation value 5 of Fig. 6. Fig. 7 shows different intersections 7 for the different interpolated derivatives of the correlations 6, which are based on normalised correlations 4 (of Fig. 6) based on different scenes (of different object properties, i.e., high to low object reflectivity or far to close object distances) as discussed above, which corresponds to the different estimated delays, i.e., the different estimated depths, for different scenes. However, the difference between scenes is very small, that is, the intersections 7 are all very close together, which is further illustrated in Fig. 8. Fig. 8 shows the residual range walk delay based on detected reflections returned from objects of different reflectivity located at the same distance according to Fig. 7. The vertical axis shows the residual range walk delays, which are based on the normalized correlation deltas 6 intersecting at intersection 7 with the x-axis of Fig. 7. The residual range walk delay corresponds to the estimated depth. The horizontal axis corresponds to the active illumination as illustrated in Fig. 1. Thus, in comparison to the range walk delay of Fig. 3 the residual range walk delay of Fig. 3 is much smaller. Therefore, also the difference between range walk delay due to reflections of different light intensities (active illumination), that is, reflections returned from objects of different reflectivity but located at the same distance or objects of the same reflectivity located at different distances, is much smaller. This shows that the estimation of depth based on the present embodiments is less sensitive to range walk errors.

[0121] Thus, the present embodiments provide a depth computation algorithm which is much less sensitive to range walk error than the classical histogram extracted features, allowing much better precision without complex characterization as illustrated by the before (Fig. 3) and after (Fig. 8) range error figures shown above.

[0122] Furthermore, the present embodiments do not require complex and potentially imprecise calibration. Also, the parameters used in the present embodiments only depend on known features of the system, for example, such as, the active illumination profile of the illuminator. Also, the algorithm lends itself to an effective hardware implementation, such as a circuitry or an electronic device, for example, an electronic device and a circuitry as illustrated in Fig. 10.

[0123] Also, to further improve the depth calculation the overall intensity of the estimated active illumination profile may be determined. For example, as illustrated in Fig. 9.

[0124] Fig. 9 illustrates the overall intensity of the estimated illumination profile and the actual overall intensity on the sensor. The horizontal axis shows the actual detected active illumination of the sensor for different scenes, which corresponds to the maximum intensity of detected reflections returned from different scenes of Fig. 1. The vertical axis shows the estimated overall intensity of the estimated active illumination profiles of different scenes, wherein the different scenes of the estimated illumination profiles are based on the same scenes of the actual illumination detected from the scenes. The estimate of the overall intensity of the active illumination profile on the sensor is based on the following: where A is the estimate of the overall active illumination intensity on the sensor, i.e., the overall intensity of the estimated active illumination profile, n is the offset for which Cn(confidence weighted correlation, see Fig. 6) is maximised, and

[0125] Hmax is the maximum value of the histogram, i.e., the acquired histogram from the macropixel.

[0126] As shown in Fig. 9, the estimated overall intensity corresponds to the actual intensity of the active illumination, which is measured on the macropixel. The purpose of Fig. 9 is to determine the amount of active light returned by the object whose depth is measured. The same effect that causes the range walk error - reduction of the proportion of active spads as the number of detected photons increases - also causes the estimation of the number of photons returned from the target to be underestimated. The above formula corrects this and may be used in a further processing step.

[0127] Fig. 10 illustrates an electronic device 20 and a circuitry 23 according to an embodiment. The electronic device 20 is an example of a (SPAD-array) dToF imaging apparatus and is configured to estimating a depth. The electronic device 20 includes a vertical-cavity surface-emitting laser (VCSEL) array 21, a macropixel array 22 and a circuitry 23.

[0128] The VCSEL array 21 is an example of an illuminator. The VCSEL array 21 is configured to emit a laser pulse 24 (which is an example of a light pulse) based on a trigger signal received by the VCSEL array 21. The laser pulse 24 includes a plurality of laser dots arranged in an array.

[0129] The VCSEL array 21 emits the laser pulse 24 into a scene that includes an object 25. The object 25 reflects (a fraction of) the laser pulse 24. The reflected laser pulse 26 travels back to the electronic device 20, passes through a lens (not shown) of the electronic device 20 and arrives at the macropixel array 22. The object 25 is an example of the object of one scene, i.e, the object at one distance or the object of one particular reflectivity. In the above examples of Figs. 1 to 9, different objects 25, i.e., objects 25 with different object properties, i.e., different reflectivity located at the same distance (high to very low) or objects of different distance with the same reflectivity, i.e., different scenes are described. Thus, the overall technique for estimating depth applies to any of the described scenes.

[0130] The macropixel array 22 is an example of a dToF sensor and includes a plurality of macropixels arranged in an array. Each macropixel of the macropixel array 22 includes a plurality of singlephoton avalanche diodes (SPADs) (which are examples of single-photon detectors). Each macropixel of the macropixel array 22 is configured to acquire a histogram (e.g., one of the histograms 1 of Fig. 2) based on a trigger signal received by the macropixel array 22, wherein the histogram indicates degrees of activation of the macropixel (i.e., a number of the SPADs of the macropixel that detect a photon) for respective sampling bins which correspond to respective subsequent time intervals, i.e., predetermined time intervals. Accordingly, the histograms acquired by the macropixel array 22 represent detected reflections 26 of respective emitted laser pulses 24 emitted by the VCSEL array 21. The histogram 1 of Fig. 2 is an example of histograms acquired by a macropixel of the macropixel array 22 for different illuminated scenes (i.e., illumination target objects of different poroperties, e.g., different objects 25, see Fig. 2 for explanation).

[0131] The circuitry 23 is configured to obtain an illumination profile of an illuminator, obtain a respective plurality of histograms from macropixels of the macropixel array 22, estimate, from the histograms, respective active illumination profiles, estimate respective confidences of the estimated active illumination profiles, and estimate a respective depth (i.e., a distance of the object 25) for each macropixel based on correlating a respective estimated active illumination profile with the obtained illumination profile of the illuminator based on the respective confidence of the estimated active illumination profile.

[0132] The circuitry 23 includes a processing unit 23a, a memory 23b and a communication unit 23c. The processing unit 23a performs data processing (including arithmetic / logic operations) necessary for estimating the depth. The memory 23b stores data necessary for the data processing by the processing unit 23a (including software instructions for the processing unit 23 a, the obtained illumination profile of the illuminator, the obtained pluralities of histograms, temporary values, estimated active illumination profiles, estimated weights, and the estimated depths etc.). The communication unit 23c receives histogram data corresponding to the plurality of histograms from the macropixel array 22 and outputs the estimated depths to an external device.

[0133] The electronic device 20 and the circuitry 23 are configured to perform any method disclosed herein, including the method of Fig. 11.

[0134] Fig. 11 illustrates a method 30 for estimating a depth according to an embodiment.

[0135] The method 30 is an example of a method performed by the electronic device 20 of Fig. 9.

[0136] At 31, the illumination profile of the VCSEL array 21 is obtained. The illumination profile may be predetermined. For example, the illumination profile may be obtained based on predetermined information, for example from the manufacturer. In other words, the illumination profile may be at the end of the production line of a dToF apparatus when calibrating the camera. The data may then be included in a memory, e.g., a flash memory, to be reused later. Thus, the illumination profile may be obtained from memory. Alternatively, the illumination profile may be generated on the fly, e.g., when using the dToF apparatus.

[0137] At 32, the VCSEL array 21 emits a laser pulse 24 into the scene, and at 33, the macropixel of the macropixel array 22 acquires the histogram. The emission of the laser pulse 24 at 32 and the acquisition of the histogram at 33 are based on (i.e., triggered by) a trigger signal. The trigger signal may be generated by a control unit (not shown in Fig. 5) of the electronic device 20 or by the circuitry 23. Examples of histograms acquired by a macropixel with 3x3 SPADs are illustrated in Fig. 2.

[0138] At 34, the circuitry 23 obtains the histogram that has been acquired by the micropixel, which is also indicated by the dashed arrow from 33 to 34.

[0139] At 35, the circuitry 23 estimates the active illumination profile based on the histogram obtained at 34. The active illumination profile may be estimated based on the difference between consecutive values of the histogram, i.e., values of subsequent adjacent sampling bins. The circuitry 23 may estimate the active illumination profile by estimating the intensity of the active illumination per time interval. Examples of estimated active illumination profiles are illustrated in Fig. 4.

[0140] At 36, the circuitry 23 estimates a confidence (weight) of the estimated active illumination profile. Examples of estimated weights for different estimated active illumination profiles are illustrated in Fig. 5.

[0141] At 37, the circuitry 23 estimates a depth (i.e., a distance of the object 25) based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator based on the confidence of the estimated active illumination profile. Correlating the estimated active illumination profile with the obtained illumination profile of the illuminator may be based on a normalized weighted (based on the estimated confidence) correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile is based on the obtainedillumination profile of the illuminator. Estimating a depth may include determining the maximum of the correlation, that is the depth may be estimated based on the maximum of the correlation. The dashed arrows from 31, 35 and 36 to 37 illustrate the input into 37.

[0142] Examples of the correlation, which may be a normalized confidence weighted correlation are illustrated in Fig. 6. The depth estimation may be further refined by basing the depth estimation on interpolating the derivative of the correlation before and after the maximum correlation. Examples of the interpolated derivatives are illustrated in Fig. 7.

[0143] Note that, although the method 30 has been described for one macropixel of the macropixel array 22, the method 30 may be performed for multiple (e.g., all) macropixels of the macropixel array 22 (e.g., simultaneously). For example, at 33, each macropixel of the macropixel array 22 may acquire a respective histogram. The circuitry may perform the processing at 34, 35 and 36 for each macropixel of the macropixel array 22 (e.g., serially or in parallel), and may thus estimate a respective depth for each macropixel of the macropixel array 22.

[0144] It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding. For example, in the method 30 of Fig. 11, the obtaining of histograms at 34 and the obtaining of the illumination profile of the illuminator may be exchanged. Also, the ordering of the estimating of the active illumination profile at 35 and the estimating of the confidence at 36 of Fig. 11 may be exchanged. Other changes of the ordering of method steps may be apparent to the skilled person.

[0145] Please note that the division of the circuitry 23 of Fig. 10 into the processing unit 23a, the memory 23b and the communication unit 23c is only made for illustration purposes and that the present disclosure is not limited to any specific division of functions in specific units. For instance, the circuitry 23 could be implemented by a respective programmed processor, field programmable gate array (FPGA) and the like.

[0146] The method 30 of Fig. 11 can also be implemented as a computer program causing a computer and / or a processor, such as circuitry 23 discussed above, to perform the method, when being carried out on the computer and / or processor. In some embodiments, also a non-transitory computer-readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the method described to be performed.

[0147] All units and entities described in this specification and claimed in the appended claims can, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such units and entities can, if not stated otherwise, be implemented by software. In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be appreciated that a computer program providing such software control and a transmission, storage or other medium by which such a computer program is provided are envisaged as aspects of the present disclosure.

[0148] Note that the present technology can also be configured as described below.

[0149] (1) A circuitry for estimating a depth, the circuitry being configured to: obtain an illumination profile of an illuminator (21); obtain a histogram (1) from a macropixel (22) detecting reflected illumination of the illuminator (21), wherein the histogram (1) indicates respective degrees of activation of the macropixel (22) in predetermined time intervals; estimate, from the histogram (1), an active illumination profile (2); estimate a confidence (3) of the estimated active illumination profile (2); and estimate a depth based on correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21), weighted by the confidence (3) of the estimated active illumination profile (2).

[0150] (2) The circuitry of (1), wherein estimating the active illumination profile (2) is based on the difference between consecutive values of the obtained histogram (1).

[0151] (3) The circuitry of (1) or (2), wherein estimating the active illumination profile (2) is based on the proportion of activation of the macropixel (22) at the start and end time of a predetermined time interval.

[0152] (4) The circuitry of any one of (1) to (3), wherein estimating the active illumination profile (2) is based on the Napierian logarithm of a ratio of a proportion of the activated macropixel (22) at the start of a predetermined time interval to a proportion of the activated macropixel (22) at the end of a predetermined time interval multiplied by the maximum value of the obtained histogram (1).

[0153] (5) The circuitry of any one of (1) to (4), wherein estimating the active illumination profile (2) is based on the proportion of the macropixel (22) that is reactivated during a time interval.

[0154] (6) The circuitry of any one of (1) to (5), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on the histogram values for each time interval. (7) The circuitry of any one of (1) to (6), wherein estimating the confidence of the estimated active illumination profile (2) is based on the geometric mean of the proportions of the activated macropixel (22) at the start and the end of a predetermined time interval.

[0155] (8) The circuitry of any one of (1) to (7), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on:

[0156] Wn=Pn.Pn.1 / (Pn+Pn.1), wherein Wnis the confidence (3) of the estimated illumination profile (2) for a predetermined time interval [n-l,n], wherein Pnis the proportion of the activated macropixel (22) at the end of the time interval [n-l,n], and wherein Pn-i is the proportion of the activated macropixel (22) at the start of the time interval [n-l,n],

[0157] (9) The circuitry of any one of (1) to (8), wherein correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21) is based on a normalized weighted correlation (4) between the estimated illumination profile (2) and a sliding illumination profile of the illuminator (21), wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator (21).

[0158] (10) The circuitry of any one of (1) to (9), wherein estimating the depth is based on a sliding filter.

[0159] (11) The circuitry of any one of (1) to (11), wherein the sliding filter is based on: wherein Cnis the normalised weighted correlation (4) for an offset of n, wherein Wn+iis the confidence of the estimated illumination profile for the [n+i-1, n+i] time interval, wherein In+iis the estimated illumination profile intensity in the [n+i-1, n+i] time interval, wherein L, is the obtained illumination profile intensity average in the [i-1, i] time interval, and wherein imaxis the duration of the obtained illumination profile of the illuminator. (12) The circuitry of any one of (1) to (11), wherein estimating the depth is based on the normalized weighted correlation (4) for which the normalized weighted correlation (4) is maximised.

[0160] (13) The circuitry of any one of (1) to (12), wherein estimating the depth is based on interpolating the derivative of the correlation (4) before and after the maximum (5) of the correlation (4).

[0161] (14) The circuitry of any one of (1) to (13), wherein an overall intensity of the estimated active illumination profile is estimated based on: wherein A is the overall intensity of the estimated active illumination profile, n is the offset for which the normalised weighted correlation (4) is maximised, and

[0162] Hmax is the maximum value of the histogram (1).

[0163] (15) An electronic device for estimating a depth, the electronic device comprising: an illuminator (21); a macropixel (22) including a plurality of single-photon detectors; and a circuitry (23) configured to: obtain an illumination profile of the illuminator (21); obtain a histogram (1) from the macropixel (22) detecting reflected illumination of the illuminator (21), wherein the histogram (1) indicates respective degrees of activation of the macropixel (22) in predefined time intervals; estimate, from the histogram (1), an active illumination profile (2); estimate a confidence (3) of the estimated active illumination profile (2); and estimate a depth based on correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21) weighted by the confidence (3) of the estimated active illumination profile (2).

[0164] (16) The electronic device of 15, wherein estimating the active illumination profile is based on the difference between consecutive values of the histogram (1). (17) The electronic device of any one of (15) to (16), wherein estimating the active illumination profile (2) is based on the proportion of activation of the macropixel (22) at the start and end time of a predetermined time interval.

[0165] (18) The electronic device of any one of (15) to (17), wherein estimating the active illumination profile (2) is based on the Napierian logarithm of a ratio of a proportion of the activated macropixel (22) at the start of a predetermined time interval to a proportion of the activated macropixel (22) at the end of a predetermined time interval multiplied by the maximum value of the obtained histogram (1).

[0166] (19) The electronic device of any one of (15) to (18), wherein estimating the active illumination profile (2) is based on the proportion of the macropixel (22) that is reactivated during a time interval.

[0167] (20) The electronic device of any one of (15) to (19), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on the histogram values for each time interval.

[0168] (21) The electronic device of any one of (15) to (20), wherein estimating the confidence of the estimated active illumination profile (2) is based on the geometric mean of the proportions of the activated macropixel (22) at the start and the end of a predetermined time interval.

[0169] (22) The electronic device of any one of (15) to (21), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on:

[0170] Wn=Pn.Pn.1 / (Pn+Pn.1), wherein Wnis the confidence (3) of the estimated illumination profile (2) for a predetermined time interval [n-l,n], wherein Pnis the proportion of the activated macropixel (22) at the end of the time interval [n-l,n], and wherein Pn-i is the proportion of the activated macropixel (22) at the start of the time interval [n-l,n],

[0171] (23) The electronic device of any one of (15) to (22), wherein correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21) is based on a normalized weighted correlation (4) between the estimated illumination profile (2) and a sliding illumination profile of the illuminator (21), wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator (21).

[0172] (24) The electronic device of any one of (15) to (23), wherein estimating the depth is based on a sliding filter.

[0173] (25) The electronic device of any one of (15) to (24), wherein the sliding filter is based on: wherein Cnis the normalised weighted correlation (4) for an offset of n, wherein Wn+iis the confidence (4) of the estimated illumination profile (2) for the [n+i-1, n+i] time interval, wherein In+iis the estimated illumination profile intensity in the [n+i-1, n+i] time interval, wherein L, is the obtained illumination profile intensity average in the [i-1, i] time interval, and wherein imaxis the duration of the obtained illumination profile of the illuminator.

[0174] (26) The electronic device of any one of (1) to (27), wherein estimating the depth is based on the normalized weighted correlation (4) for which the normalized weighted correlation (4) is maximised.

[0175] (27) The electronic device of any one of (15) to (26), wherein estimating the depth is based on interpolating the derivative of the correlation (4) before and after the maximum (5) of the correlation.

[0176] (28) The electronic device of any one of (15) to (27), wherein an overall intensity of the estimated active illumination profile is estimated based on: wherein A is the overall intensity of the estimated active illumination profile (2), n is the offset for which the normalised weighted correlation is maximised, and

[0177] Hmax is the maximum value of the histogram (1).

[0178] (29) A method for estimating a depth, the method comprising: obtaining an illumination profile of an illuminator (21); obtaining a histogram (1) from a macropixel (22) detecting reflected illumination of the illuminator (21), wherein the histogram (1) indicates respective degrees of activation of the macropixel (22) in predefined time intervals; estimating, from the histogram (1), an active illumination profile (2); estimating a confidence (3) of the estimated active illumination profile (2); and estimating a depth based on correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21) weighted by the confidence (3) of the estimated active illumination profile (2).

[0179] (30) The method of (29), wherein estimating the active illumination profile (2) is based on the difference between consecutive values of the obtained histogram (1).

[0180] (31) The method of (29) or (30), wherein estimating the active illumination profile (2) is based on the proportion of activation of the macropixel (22) at the start and end time of a predetermined time interval.

[0181] (32) The method of any one of (29) to (31), wherein estimating the active illumination profile (2) is based on the Napierian logarithm of a ratio of a proportion of the activated macropixel (22) at the start of a predetermined time interval to a proportion of the activated macropixel (22) at the end of a predetermined time interval multiplied by the maximum value of the obtained histogram (1).

[0182] (33) The method of any one of (29) to (32), wherein estimating the active illumination profile (2) is based on the proportion of the macropixel (22) that is reactivated during a time interval.

[0183] (34) The method of any one of (29) to (33), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on the histogram values.

[0184] (35) The method of any one of (29) to (34), wherein estimating the confidence of the estimated active illumination profile (2) is based on the geometric mean of the proportions of the activated macropixel (22) at the start and the end of a predetermined time interval.

[0185] (36) The method of any one of (29) to (35), wherein estimating the confidence (3) of the estimated active illumination profile (2) is based on:

[0186] Wn=Pn.Pn-1 / (Pn+Pn.1), wherein Wnis the confidence (3) of the estimated illumination profile (2) for a predetermined time interval [n-l,n], wherein Pnis the proportion of the activated macropixel (22) at the end of the time interval [n-l,n], and wherein Pn-i is the proportion of the activated macropixel (22) at the start of the time interval [n-l,n],

[0187] (37) The method of any one of (29) to (36), wherein correlating the estimated active illumination profile (2) with the obtained illumination profile of the illuminator (21) is based on a normalized weighted correlation (4) between the estimated illumination profile (2) and a sliding illumination profile of the illuminator (21), wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator (21).

[0188] (38) The method of any one of (29) to (37), wherein estimating the depth is based on a sliding filter.

[0189] (39) The method of any one of (29) to (38), wherein the sliding filter is based on: wherein Cnis the normalised weighted correlation (4) for an offset of n, wherein Wn+iis the confidence of the estimated illumination profile for the [n+i-1, n+i] time interval, wherein In+iis the estimated illumination profile intensity in the [n+i-1, n+i] time interval, wherein L, is the obtained illumination profile intensity average in the [i-1, i] time interval, and wherein imaxis the duration of the obtained illumination profile of the illuminator.

[0190] (40) The method of any one of (29) to (39), wherein estimating the depth is based on the normalized weighted correlation (4) for which the normalized weighted correlation (4) is maximised.

[0191] (41) The method of any one of (29) to (40), wherein estimating the depth is based on interpolating the derivative of the correlation (4) before and after the maximum (4) of the correlation (4). (42) The method of any one of (29) to (41), wherein an overall intensity of the estimated active illumination profile is estimated based on: wherein A is the overall intensity of the estimated active illumination profile, n is the offset for which the normalised weighted correlation (4) is maximised, and

[0192] Hmax is the maximum value of the histogram (1).

[0193] (43) A computer program comprising program code causing a computer to perform the method according to anyone of (29) to (42), when being carried out on a computer.

[0194] (44) A non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the method according to anyone of (29) to (42) to be performed.

Claims

CLAIMS1. A circuitry for estimating a depth, the circuitry being configured to: obtain an illumination profile of an illuminator; obtain a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predetermined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

2. The circuitry of claim 1, wherein estimating the active illumination profile is based on the difference between consecutive values of the obtained histogram.

3. The circuitry of claim 1, wherein estimating the active illumination profile is based on the proportion of activation of the macropixel at the start and end time of a predetermined time interval.

4. The circuitry of claim 3, wherein estimating the active illumination profile is based on the Napierian logarithm of a ratio of a proportion of the activated macropixel at the start of a predetermined time interval to a proportion of the activated macropixel at the end of a predetermined time interval multiplied by the maximum value of the obtained histogram.

5. The circuitry of claim 1, wherein estimating the active illumination profile is based on the proportion of the macropixel that is reactivated during a time interval.

6. The circuitry of claim 1, wherein estimating the confidence of the estimated active illumination profile is based on the histogram values for each time interval.

7. The circuitry of claim 6, wherein estimating the confidence of the estimated active illumination profile is based on:Wn=Pn.Pn.1 / (Pn+Pn.1), wherein Wnis the confidence of the estimated illumination profile for a predetermined time interval [n-l,n],wherein Pnis the proportion of the activated macropixel at the end of the time interval [n-I,n], and wherein Pn-i is the proportion of the activated macropixel at the start of the time interval [n-l,n],8. The circuitry of claim 1, wherein correlating the estimated active illumination profile with the obtained illumination profile of the illuminator is based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator.

9. The circuitry of claim 1, wherein estimating the depth is based on a sliding filter.

10. The circuitry of claim 1, wherein estimating the depth is based on a normalized weighted correlation for which the normalized weighted correlation is maximised.I I . The circuitry of claim 1, wherein the sliding filter is based on:wherein Cnis the normalised weighted correlation for an offset of n, wherein Wn+iis the confidence of the estimated active illumination profile for the [n+i-1, n+i] time interval, wherein In+iis the estimated active illumination profile intensity in the [n+i-1, n+i] time interval, wherein L, is the obtained illumination profile intensity average in the [i-1, i] time interval, and wherein imaxis the duration of the obtained illumination profile of the illuminator.

12. The circuitry of claim 1, wherein estimating the depth is based on interpolating the derivative of the correlation before and after the maximum of the correlation.

13. The circuitry of claim 1, wherein an overall intensity of the estimated active illumination profile is estimated based on:wherein A is the overall intensity of the estimated active illumination profile, n is the offset for which the normalized weighted correlation is maximised, andHmax is the maximum value of the histogram.

14. An electronic device for estimating a depth, the electronic device comprising: an illuminator; a macropixel including a plurality of single-photon detectors; and a circuitry configured to: obtain an illumination profile of the illuminator; obtain a histogram from the macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals; estimate, from the histogram, an active illumination profile; estimate a confidence of the estimated active illumination profile; and estimate a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

15. The electronic device of claim 14, wherein estimating the active illumination profile is based on the difference between consecutive values of the histogram.

16. The electronic device of claim 14, wherein estimating the confidence of the estimated active illumination profile is based on the histogram values.

17. The electronic device of claim 14, wherein correlating the estimated active illumination profile with the obtained illumination profile of the illuminator is based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator.

18. A method for estimating a depth, the method comprising: obtaining an illumination profile of an illuminator; obtaining a histogram from a macropixel detecting reflected illumination of the illuminator, wherein the histogram indicates respective degrees of activation of the macropixel in predefined time intervals;estimating, from the histogram, an active illumination profile; estimating a confidence of the estimated active illumination profile; and estimating a depth based on correlating the estimated active illumination profile with the obtained illumination profile of the illuminator weighted by the confidence of the estimated active illumination profile.

19. The method of claim 18, wherein correlating the estimated active illumination profile with the obtained illumination profile of the illuminator is based on a normalized weighted correlation between the estimated illumination profile and a sliding illumination profile of the illuminator, wherein the sliding illumination profile is based on the obtained illumination profile of the illuminator.

20. The method of claim 18, wherein estimating the active illumination profile is based on the proportion of activation of the macropixel at the start and end time of a predetermined time interval.

Images