Method and system for uniforming pixel values in a pixel array

Abstract

The invention relates to a method and system for uniformizing pixel values in a pixel array. A method for homogenizing values of an array of pixels includes identifying pixels for which a target has been detected and correctable hole pixels for which no target has been detected, the correctable hole pixels being surrounded by at least one adjacent pixel for which the target has been detected; calculating an equivalent target detection confidence indicator for each correctable void pixel according to a target feature value calculated for adjacent pixels of the correctable void pixel, the equivalent target detection confidence indicator represents that a target detected in a neighborhood according to a parameter including the sensitivity of a hole pixel also exists at a position of a correctable hole pixel, and when the calculated equivalent target detection confidence indicator is below a detection threshold value, calculating a corrected target feature value associated with the hole pixel, wherein a target detected in the neighborhood will have a target detection confidence indicator at the correctable hole pixel.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of European Patent Application No. 24305612, filed April 19, 2024, which is incorporated herein by reference. TECHNICAL FIELD

[0003] Some embodiments of the present disclosure relate to “time-of-flight” (ToF) devices (dToF: direct time-of-flight or iToF: indirect time-of-flight devices) and more specifically to homogenization of pixel values (typically distances) in a pixel array. BACKGROUND

[0004] Devices for determining the distance (or depth, range) of an object or target are known. The method currently used is called “time-of-flight” (ToF). This method comprises sending a light signal to an object and measuring the time required for the signal to propagate to the object and return to the device.

[0005] dToF (direct time-of-flight) devices directly measure the time required for a signal to be transmitted to an object and return to the device.

[0006] iToF (indirect time-of-flight) devices calculate the time required for a signal to propagate by measuring the phase shift between the signal from the light source and the signal reflected from the object and detected by a photosensor. Knowing this phase shift and the speed of light makes it possible to determine the distance to the object.

[0007] Single-photon avalanche diodes (SPADs) can be used as detectors of reflected light pulses. Photons can generate charge carriers in a SPAD through the photoelectric effect. The photo-generated charge carriers can trigger an avalanche current in one or more SPADs in a SPAD pixel array. The avalanche current can be indicative of a certain event, i.e. a photon has been detected.

[0008] In order to produce precise time-of-arrival information about each photon from the optical light radiation, a single-photon sensitive detector can be configured to generate a time-series histogram of the number of detected photons during a continuous emission. This time-series histogram is thus composed of a plurality of bins, each bin associating a number of detected photons with a given acquisition time during the emission. An algorithm is then implemented to identify the bins of the histogram that represent photons resulting from the reflection of radiation by one or more objects.

[0009] In high-resolution ToF sensors (e.g. sensors used for facial identification applications), a SPAD pixel array is provided to acquire a depth map of a scene in the field of view. The array can contain tens or hundreds of pixels. Each pixel can contain a set of multiple SPAD detectors.

[0010] The sensitivity profile of a pixel can not be uniform along the array of the sensor. The sensitivity of each pixel can depend on the number of functional SPADs in a plurality of SPAD groups. For example, the sensitivity of a pixel containing 2 functional SPADs (out of a group of 4) will be half the sensitivity of a pixel containing all 4 functional SPADs. In the worst case, a pixel can be completely dead, with no functional SPADs.

[0011] Statistically, defects and malfunctions exist in SPADs, for example due to random physical drifts in the manufacturing method steps.

[0012] In addition, the variation of the dark current of each respective SPAD due to random physical drifts can also reduce the sensitivity of a pixel with respect to another pixel.

[0013] In addition, the variation of the ambient noise level of each respective SPAD due to the scene can also reduce the sensitivity of a pixel with respect to another pixel.

[0014] Therefore, a target covering the entire field of view can exceed the detection threshold of certain pixels and not of others, which will lead to the target being falsely detected with holes. Hole pixels can be defined as pixels that do not detect a target or for which the value of the detected target feature is not defined, surrounded by pixels that do detect a target and associated with a computed value of the target feature.

[0015] When holes appear in a target, the problem is to determine whether these holes are real or not, in order to correct them (“fill the holes”) or not. This determination process should advantageously be embedded in the ToF sensor firmware.

[0016] The classical and well-known solution is to fill the holes systematically, without considering the possibility that the target contains real holes. The correction to fill the holes is usually performed by interpolation from the values of the neighboring pixels.

[0017] Therefore, there is a need to provide an embedded solution to accurately decide whether to fill the holes or not. SUMMARY

[0018] According to embodiments, a homogenization method and system are proposed, with the following advantages:

[0019] It can mitigate the sensitivity profile of certain pixels of a ToF sensor.

[0020] It allows to increase the maximum range of certain pixels.

[0021] It can correct bad pixels.

[0022] It is implemented in an embedded way in the signal processing of the firmware, generating a distance map output.

[0023] It does not create latency issues in the computation.

[0024] According to an embodiment, it is proposed that,

[0025] - analyzing the targets of the hole neighborhood, if any,

[0026] - for each target (in the same distance group), analyzing that it has a pattern,

[0027] - filtering out those patterns that a priori do not need to be filled in the hole (typically because the underlying fact ("real" target) of the known association is unlikely to appear, or because of a specific use case),

[0028] - for the remaining patterns (which were not filtered out), computing an equivalent target detection confidence indicator at the hole pixel, defined as "the target detection confidence indicator that a target detected in the neighborhood would have at the hole pixel, and computed from the target feature values of the neighboring pixels, when considering the sensitivity of the hole pixel, if the target present in the neighborhood was also present at the hole pixel location",

[0029] - deciding not to fill in the hole when the equivalent target detection confidence indicator is greater than a detection threshold and thus if the target would have been likely detected if it was also present at the hole pixel location in the underlying fact.

[0030] - else, when the equivalent target detection confidence indicator is not greater than the detection threshold and if the target would not have been likely detected if it was also present at the hole pixel location in the underlying fact, deciding to fill in the hole, for example by interpolation based on the values of the neighboring pixels.

[0031] According to one aspect, it is proposed a method for homogenizing values of a pixel array, the method comprising:

[0032] - identifying pixels for which a target has been detected and correctable hole pixels for which no target has been detected, the hole pixels being surrounded by at least one neighboring pixel for which a target has been detected,

[0033] - computing target feature values for the pixels for which a target has been detected, computing for each correctable hole pixel an equivalent target detection confidence indicator from the target feature values computed for the neighboring pixels of the correctable hole pixel, the equivalent target detection confidence indicator representing a neighborhood target detection confidence indicator: the target detected in the neighborhood of the respective correctable hole pixel would have had the neighborhood target detection confidence indicator at the respective correctable hole pixel, according to parameters including the sensitivity of the respective correctable hole pixel, as if the target detected in the neighborhood was also present at the location of the respective correctable hole pixel,

[0034] - calculating a corrected target feature value associated with the hole pixel when the calculated equivalent target detection confidence indicator is lower than a detection threshold.

[0035] A target feature value can be defined as a direct or indirect property of the target computed from the signal sensed by the array of pixels. For example, in the dToF case, the target feature values can include the range, the useful signal rate and the target detection confidence indicator computed from the generated histogram.

[0036] The target detection confidence indicator is for example computed to provide an indication that the target is detected with a sufficient reliability or plausibility with respect to the accuracy of the time-of-flight detection.

[0037] According to an embodiment, the parameters used to compute said equivalent target detection confidence indicator include the useful signal rate of at least one neighboring pixel.

[0038] According to an embodiment, the parameters used to compute said equivalent target detection confidence indicator include the ambient noise rate of the hole pixel and the ambient noise rate of at least one neighboring pixel of the neighboring pixels.

[0039] According to an embodiment, the parameters used to compute the equivalent target detection confidence indicator include the number of active single-photon detectors in the hole pixel and the integration time.

[0040] According to an embodiment, the parameters used to compute said equivalent target detection confidence indicator include the crosstalk rate of the hole pixel and the crosstalk rate of at least one neighboring pixel.

[0041] According to an embodiment, identifying said correctable hole pixels comprises identifying in the set of position patterns, a pattern of neighboring pixels of the respective hole pixel and said target feature values at the neighboring pixels representative of a correctable hole pixel situation.

[0042] According to an embodiment, the neighboring pixels of a correctable hole pixel are pixels in a lateral or diagonal contact position with the hole pixel.

[0043] According to an embodiment, the method further comprises:

[0044] - emitting light radiation;

[0045] - detecting, by the pixels of the array, photons of the light radiation,

[0046] - generating a histogram comprising bins associating the number of detected photons with a given acquisition time per pixel,

[0047] - processing the histograms of said pixels to provide an identification of pixels having detected a target, correctable hole pixels and target feature values.

[0048] According to an embodiment, the target detection confidence indicator is configured to trigger a binary decision, i.e. a target is decided to have been detected if the value of the indicator is greater than a threshold, or a target is decided to not have been detected if the value of the indicator is lower than the threshold.

[0049] According to another aspect, there is presented a system comprising a pixel array and a processor configured to:

[0050] - identify pixels for which a target has been detected and hole pixels for which no target has been detected, the hole pixels being surrounded by at least one neighboring pixel for which a target has been detected,

[0051] - compute target feature values for the pixels for which a target has been detected, compute for each hole pixel for which no target has been detected an equivalent target detection confidence indicator, the equivalent target detection confidence indicator representing a neighborhood target detection confidence indicator: a target would have had the neighborhood target detection confidence indicator at the respective hole pixel if the target had been detected in a neighborhood of the respective hole pixel, according to parameters comprising a sensitivity of the respective hole pixel,

[0052] - compute a corrected target feature value associated with a hole pixel when the computed equivalent target detection confidence indicator is lower than a detection threshold.

[0053] According to an embodiment, the processor is configured to compute the equivalent target detection confidence indicators according to the computing steps of the method defined above.

[0054] According to an embodiment, the processor is configured to identify the hole pixels according to the identifying steps of the method defined above.

[0055] According to an embodiment, the system further comprises:

[0056] - an emitter configured to be able to emit light radiation,

[0057] - a receiver comprising a pixel array and configured to be able to detect photons of the light radiation,

[0058] - wherein the processor is configured to generate a histogram comprising bins associating a number of detected photons with a given acquisition time for each pixel, and

[0059] - wherein the processor is configured to process the histogram of pixels to provide the identification of the pixels for which a target has been detected, the hole pixels for which no target has been detected and the target feature values. BRIEF DESCRIPTION OF DRAWINGS

[0060] Other advantages and features of the present application will become apparent after a review of the detailed description of non-limiting implementations and embodiments, and the appended claims, taken in conjunction with the accompanying drawings, wherein:

[0061] Figure 1 Fig. illustrates a time-of-flight sensor SENS according to an embodiment;

[0062] Figure 2 Fig. illustrates a method for homogenizing target feature values;

[0063] Figure 3 Fig. illustrates an example set of patterns representing correctable cases; and

[0064] Figure 4 Fig. illustrates adjacent pixel lateral or diagonal position contact hole pixels. DETAILED DESCRIPTION

[0065] Figure 1 Fig. illustrates a time-of-flight sensor SENS according to an embodiment.

[0066] The time-of-flight sensor SENS comprises an emitter ME configured to periodically emit light radiation RE.

[0067] The emitter ME can consist of a vertical-cavity surface-emitting laser, usually abbreviated by those skilled in the art as "VCSEL".

[0068] The time-of-flight sensor SENS can receive reflected light radiation RR resulting from the reflection of the light radiation on the object OBJ if one or more objects OBJ are present within the field of light radiation.

[0069] The time-of-flight sensor SENS thus comprises a receiver MR configured to receive the light radiation RR reflected by the object OBJ within the field of view of the time-of-flight sensor.

[0070] The receiver MR comprises a photonic detector, for example a single-photon avalanche diode, usually referred to by those skilled in the art as "SPAD", which can act as a detector of the reflected light pulses. The SPAD photonic detector can be arranged in a pixel array, capable of acquiring a depth map of the scene within the field of view. The array can comprise tens or hundreds of pixels, for example an array of 16*16 = 256 pixels. Each pixel can comprise a group of multiple SPAD detectors, for example each pixel can comprise a group of 4 SPAD detectors.

[0071] The time-of-flight sensor SENS comprises a histogram generator MGH configured to generate a histogram from the signals output by the array of single-photon detectors. In particular, the histogram generator MGH is configured to count the number of photons detected by the receiver at a plurality of consecutive acquisition times.

[0072] The histogram generator MGH is thus configured to generate a histogram comprising different bins. Each bin associates a number of detected photons with a given acquisition time.

[0073] More specifically, a histogram is acquired over a given period of time. The bins of the histogram are associated with different acquisition times of the histogram acquisition period. An acquisition period of a histogram starts when the emitter emits light radiation and lasts until a predefined number of bins is acquired.

[0074] Some bins of such a histogram can represent photons from radiation reflected by an object and thus the presence of an object within the field of view of the sensor.

[0075] An algorithm is then implemented to identify the bins of the histogram that represent photons resulting from radiation reflected by one or more objects.

[0076] In particular, the bins representing the presence of an object can be identified by comparing the bins to a fixed threshold greater than or equal to a given threshold that defines the ambient noise. The ambient noise generally refers to the background noise resulting from the lighting conditions used by the sensor (in a typical ToF application, it refers to the infrared "ambient" light not emitted by the emitter).

[0077] The "time" here should be understood as a very short duration compared to the acquisition period of the histogram. For example, the acquisition time can last 250 picoseconds and the acquisition period of the histogram can last 36 nanoseconds to acquire 144 bins.

[0078] Moreover, a plurality of histograms can also be acquired successively. The overall period of acquisition of these histograms can be in the range of 15 milliseconds.

[0079] Preferably, the histogram generator MGH is implemented by software, code or instructions executed by the processor UT of the time-of-flight sensor. For example, the processor UT can be integrated in a microprocessor device.

[0080] The time-of-flight sensor SENS also comprises a processor MT configured to post-process the generated histogram. The post-processing allows to determine the bins representing the presence of an object within the field of view of the sensor or, in other words, to identify the pixels for which a target has been detected and the pixels for which a target has not been detected. The bins representing the presence of an object allow to determine the distance between the object and the sensor by considering the acquisition time associated with this bin.

[0081] Preferably, the processor MT is implemented by software, code or instructions executed by the processor UT.

[0082] In particular, the processing MT is configured to process the histograms of pixels in order to provide processed values conveying detection information and properties of the presence of a target in the field of view at the respective pixel position. These processed values are referred to as "target feature values" from the respective pixels.

[0083] The target feature values can be defined as direct or indirect properties of the target computed from the signals sensed by the pixel array. For example, in the dToF case, the target feature values can include range, useful signal rate and target detection confidence indicator computed from the generated histograms.

[0084] Moreover, the processor MT is configured to perform the method 200 for homogenizing the target feature values described hereafter with respect to Figure 2 The method 200 for homogenizing the target feature values is described hereafter.

[0085] The homogenization method 200 can be performed in the pipeline processing of the ToF sensor SENS, as signal processing between the acquisition of "raw" frames by the pixel array and the output of the target feature map (e.g. distance map).

[0086] Therefore, reference is now made to Figure 2 .

[0087] Figure 2 The method 200 for homogenizing the target feature values is described hereafter.

[0088] A "hole pixel" is defined as a pixel that has not detected a target and is therefore associated with no defined target feature value, this pixel being surrounded by pixels that have detected a target, that is to say pixels associated with a target feature value.

[0089] The method 200 is performed in the full pipeline processing performed by the ToF sensor SENS, after a stage 201 that provides target feature values from the pixels of the array. The target feature values are for example based on the processing of "raw" (histogram) signals in order to convey information of the detection of the presence of a target in the field of view at the position of the respective pixel.

[0090] As previously described with respect to Figure 1 The raw signals can be acquired by performing the following steps: emitting optical radiation; detecting photons of the optical radiation by the pixel array; generating a histogram comprising bins associating the number of detected photons with a given acquisition time per pixel; processing the histograms of pixels to provide the identification of pixels that have detected a target or not and target feature values.

[0091] The method then comprises an identification step 210 configured to identify correctable hole pixels, that is to say hole pixels that can be corrected.

[0092] The identifying step 210 can comprise analyzing the targets, if any, in the pixels neighboring the hole pixel, in order to filter out 211 those hole pixels for which the hole does not a priori require correction (for example because the associated underlying facts are not likely to occur or are due to a particular use case).

[0093] With reference to Figure 4 , the neighboring pixels px_i are the pixels in the positions laterally or diagonally in contact with the hole pixel px_k, that is to say the 8 pixels px_i of the 3*3 square array surrounding the central pixel px_k.

[0094] In the simplest implementation, the identifying step 210 can filter out the holes having less than a threshold number of targets detected in the neighboring pixels px_i.

[0095] In an advantageous implementation, the identifying step 210 comprises identifying the pattern 311-319, 320 of the neighborhood of the respective hole pixel px_k in the set of patterns 310, 320 representing the cases that should not be filtered out, i.e. the correctable cases. Figure 3 ). Figure 3 The patterns can be determined as a function of the position and of the target characteristic value in the neighboring pixels px_i. Specifically, for a given neighboring pixel px_i, there can be multiple patterns corresponding to different targets grouped by their distance from the sensor.

[0096] Reference is now made to this with respect to

[0097] . Figure 3

[0098] Figure 3 An example set of patterns 310, 320 representing correctable cases is illustrated.

[0099] For example, the patterns can be stored in a non-transitory internal memory MEM accessible by the processor UT and the processing MT. Software, code or instructions for performing the actions described herein can also be stored in the non-transitory internal memory MEM for execution by the processor UT.

[0100] The patterns are based on the position and on the target characteristic value of the neighboring pixels around the hole pixel. In each of the pattern examples, the hole pixel is represented by a thick white box, the neighboring pixels for which a target is detected and for which a target characteristic value is therefore generated are represented by cross-hatched boxes, and the neighboring pixels for which no target is detected and for which an undefined target characteristic value is therefore generated are represented by thin white boxes.

[0101] The set of position patterns 310 illustrates a "normal" case, in which the hole pixel is not located at a corner of the array and is therefore surrounded by 8 pixels in the 3*3 square local sub-array.

[0102] ​Line 311 illustrates the case where all 8 neighboring pixels around the hole pixel have generated a target feature value belonging to the group of interest (typically the target distance values of the 8 neighboring pixels around the hole pixel are of the same order of magnitude).

[0103] Line 312 illustrates the case where 7 out of 8 neighboring pixels around the hole pixel have generated a target feature value.

[0104] Line 313 illustrates the case where 6 neighboring pixels have generated a target feature value, including 3 pixels on the side and 3 pixels on the diagonal.

[0105] Line 314 illustrates the case where 6 neighboring pixels have generated a target feature value, including 4 pixels on the side and 2 pixels on the diagonal belonging to the same row or column.

[0106] Line 315 illustrates the case where 5 neighboring pixels have generated a target feature value, belonging to two adjacent rows or columns.

[0107] Line 316 illustrates the case where 4 neighboring pixels have generated a target feature value, including 3 pixels on the side and 1 pixel on the diagonal, adjacent to 2 of the 3 pixels on the side.

[0108] Line 317 illustrates the case where only all 4 neighboring pixels on the side have generated a target feature value.

[0109] Line 318 illustrates the case where 3 out of 4 neighboring pixels on the side have generated a target feature value.

[0110] Line 319 illustrates the case where 2 out of 4 neighboring pixels on the side or on the diagonal have generated a target feature value, these 2 pixels being on the same row, column or diagonal.

[0111] The set of position patterns 320 illustrates the case where the hole pixel is located at the corner of the array and all neighboring pixels around the hole pixel have generated a target feature value.

[0112] Now coming back to Figure 2 Figure 2 .

[0113] The method 200 then comprises, for the remaining patterns 212 that have not been filtered out, a step 220 of calculating an equivalent target detection confidence indicator at the hole pixel px_k, using as calculation input the target feature values computed from the surrounding pixels px_i, the targets detected in the neighborhood would have had these target feature values at the hole pixel when considering that the target was also present at the hole pixel location, taking into account some parameters including the sensitivity of the hole pixel.

[0114] The target detection confidence indicator is configured to trigger a binary decision, i.e. either a target is detected at this pixel (if the indicator is above a threshold) or no target is detected at this pixel (if the indicator is below a threshold).

[0115] For example, the target detection confidence indicator can generally be a statistical measure commonly used in the technical field to express a signal-to-noise ratio, such as the measure called "Z-score" (defined in European patent application publication EP 4063899 A1, as described below).

[0116] In other words, step 220 comprises computing an equivalent target detection confidence indicator at each correctable hole pixel 212, which equivalent target detection confidence indicator represents the target detection confidence indicator that a target detected in the neighborhood would have at the hole pixel, when the target detected in the neighborhood, according to the relevant target feature values computed in the neighboring pixels px_i of the correctable hole pixel px_k, and according to the parameters including the sensitivity of the hole pixel, would also be present at the hole pixel location.

[0117] Then, when the computed equivalent target detection confidence indicator is greater than the detection threshold of the photodetector and thus likely to detect the target if it would also be present at the hole pixel location in the underlying reality, it is decided not to fill the hole 221.

[0118] Otherwise, when the computed equivalent target detection confidence indicator is not greater than the detection threshold and thus likely not to detect the target if it would also be present at the hole pixel location in the underlying reality, it is decided to fill the hole 222.

[0119] The "filling of the hole", that is to say the computation of the corrected target feature value associated with the (corrected) hole pixel and the replacement of the undefined target feature value, can be performed for example by a classical interpolation operation based on the values of the neighboring target feature values.

[0120] For example, the interpolation operation can essentially average the values of the neighboring target feature values, or compute other classical calculations.

[0121] The computation of the equivalent target detection confidence indicator that a target detected in the neighborhood would have at the hole pixel, if the target would also be present at the hole pixel location, is based on the available computation parameters from the hole pixel px_k and the neighboring pixels, such as device parameters and target feature values.

[0122] Thus, these computation parameters can include target feature values, such as the target detection confidence indicator z i of the (at least one) neighboring pixel px_i the ambient rate of the hole pixel and the ambient rate of at least one of the neighboring pixels

[0123] ambient rate of the pixel Usually refers to the background noise originating from the sensor using the lighting conditions (in a typical ToF application, to the infrared "ambient" light not emitted by the emitter).

[0124] Moreover, the sensitivity parameters can comprise device and control parameters, such as the number of active single-photon detectors in the hole pixel N k and their bin integration time tT k , advantageously related to the number of active single-photon detectors in the neighboring pixels N i and their respective bin integration time tT i .

[0125] Moreover, the crosstalk rate of the hole pixel and the crosstalk rate of the neighboring pixels can be assumed equal if not explicitly known.

[0126] crosstalk rate of the pixel Usually refers to the unwanted parasitic signals emitted by the emitter ME, such as a VCSEL, and directly propagating inside the time-of-flight device to the receiver MR, such as to the SPAD pixel array of the time-of-flight sensor SENS.

[0127] For example, in the case of a "dToF" (traditionally for "direct time-of-flight") device, the target detection confidence indicator can be the Z-score defined in European patent application publication EP 4 063 899 Al. An equivalent Z-score can be obtained starting from the following expression of the Z-score:

[0128]

[0129] and assuming that the target completely covers the area px_i of the neighboring pixels and the area px_k of the hole pixel, without tilt, that is:

[0130]

[0131] and also assuming that the crosstalk rates of the neighboring pixels and of the hole pixel are equal, that is:

[0132]

[0133] the final expression of the equivalent Z-score is obtained:

[0134]

[0135] Equations [Equation 1] to [Equation 5] use the following notations (index "j" denotes index "i" for a neighboring pixel px_i or index "k" for a hole pixel px_k):

[0136] N j : number of SPADs in a pixel;

[0137] T j : total time spent;

[0138] t: time interval duration;

[0139] ambient rate (in counts per second and per SPAD) (shear value is thus never null);

[0140] useful signal rate (in counts per second and per SPAD);

[0141] cross-talk rate (in counts per second and per SPAD);

[0142] n, b: coefficients.

[0143] Multiple calculations of [Equation 5] can be made on multiple neighboring pixels px_i and the equivalent Z-score for a hole pixel px_k can for example be the average of all calculated Z-scores of [Equation 5] z k .

[0144] As an alternative embodiment to directly "filling the hole" if the equivalent target detection confidence indicator obtained is not greater than the detection threshold 222, some a priori knowledge of the target faced by the sensor can be used to make the decision after the optional step 230.

[0145] For example, the a priori knowledge can provide a probability score for a first hypothesis "H1" that the target is present in the hole pixel px_k detection area and a probability score for a second hypothesis "H2" that the target is not present in the hole pixel px_k detection area.

[0146] Indeed, even if the equivalent target detection confidence indicator is less than the detection threshold, it is still possible that the probability H2 is greater than the probability H1.

[0147] Such probability scores can for example be computed from a set of previous acquisitions of the same target. For example, consideration of past acquisitions can be used to detect "flickering" effects of pixels below the detection threshold.

[0148] Thus, if the probability H1 is greater than the probability H2, it is decided 231 to fill the hole px_k; whereas if the probability H1 is less than the probability H2, it is decided 232 not to fill the hole px_k.

[0149] In summary, the present disclosure discloses systems and methods and provides an embedded solution that accurately determines whether a hole is real or not in order to correct it by using an equivalent target detection confidence indicator and a neighborhood pattern to infer the reality of the target at pixel positions for which the target feature value is undefined. The disclosed embodiments and implementations provide the following advantages: mitigating the distribution of sensitivity of some pixels of a ToF sensor; increasing the maximum range of some pixels; correcting bad pixels; not generating a latency problem; and being implemented in a fully embedded manner.

Claims

1. A method for homogenizing values of a pixel array, the method comprising: identifying pixels that have detected a target and correctable hole pixels that have not detected a target, the correctable hole pixels being surrounded by at least one neighboring pixel that has detected a target; computing a target feature value for the pixels that have detected the target; computing, for each correctable hole pixel, an equivalent target detection confidence indicator from the target feature values computed for the neighboring pixels of the correctable hole pixel, the equivalent target detection confidence indicator representing a neighborhood target detection confidence indicator: from parameters including sensitivity of the respective correctable hole pixel, detecting the target in a neighborhood of the respective correctable hole pixel would have the neighborhood target detection confidence indicator at the respective correctable hole pixel as if the target detected in the neighborhood also existed at the location of the respective correctable hole pixel; and in response to the computed equivalent target detection confidence indicator being below a detection threshold, computing a corrected target feature value associated with the respective correctable hole pixel.

2. The method of claim 1, wherein the parameters used to compute the equivalent target detection confidence indicator include a useful signal rate of at least one of the neighboring pixels.

3. The method of claim 1, wherein the parameters used to compute the equivalent target detection confidence indicator include an ambient noise rate of the respective correctable hole pixel and an ambient noise rate of at least one of the neighboring pixels.

4. The method of claim 1, wherein the parameters used to compute the equivalent target detection confidence indicator include a number of active single photon detectors in the respective correctable hole pixel and an integration time.

5. The method of claim 1, wherein the parameters used to compute the equivalent target detection confidence indicator include a crosstalk rate of the respective correctable hole pixel and a crosstalk rate of at least one of the neighboring pixels.

6. The method of claim 1, wherein identifying the correctable hole pixels includes identifying a pattern in the neighboring pixels of the respective corrected hole pixel in a set of position patterns and the target feature values at the neighboring pixels representing a correctable hole pixel situation.

7. The method of claim 1, wherein the neighboring pixels of the respective correctable hole pixel are pixels that are laterally or diagonally in contact with the respective correctable hole pixel.

8. The method of claim 1, further comprising: emitting optical radiation; detecting photons of the optical radiation by the pixel array; generating a histogram including bins associating a number of detected photons with a given acquisition time for each pixel; and processing the histogram of the pixels to provide the identification of pixels that have detected the target, the correctable hole pixels, and the target feature values.

9. The method of claim 1, wherein the neighborhood target detection confidence indicator is configured to trigger a binary decision: if the value of the indicator is greater than a threshold value, then a determination is made that the target has been detected, or if the value of the indicator is lower than the threshold value, then a determination is made that no target has been detected.

10. A system comprising: a pixel array; and a processor coupled to the pixel array and configured to: identify pixels for which a target has been detected and correctable hole pixels for which no target has been detected, the correctable hole pixels being surrounded by at least one neighboring pixel for which the target has been detected; calculate target feature values for the pixels for which the target has been detected; calculate, for each correctable hole pixel, an equivalent target detection confidence indicator from the target feature values calculated for the neighboring pixels of the correctable hole pixel, the equivalent target detection confidence indicator representing a neighborhood target detection confidence indicator: detection of the target in a neighborhood of the respective correctable hole pixel would have the neighborhood target detection confidence indicator at the respective correctable hole pixel according to parameters including sensitivity of the respective correctable hole pixel as if the target detected in the neighborhood also existed at the location of the respective correctable hole pixel; and in response to the calculated equivalent target detection confidence indicator being lower than a detection threshold, calculate a corrected target feature value associated with the respective correctable hole pixel.

11. The system of claim 10, wherein the parameters used to calculate the equivalent target detection confidence indicator include a useful signal rate of at least one of the neighboring pixels.

12. The system of claim 10, wherein the processor configured to identify the correctable hole pixels includes a processor configured to identify a pattern in the neighboring pixels of the respective correctable hole pixel in a set of location patterns and a target feature value at the neighboring pixels representing a correctable hole pixel situation.

13. The system of claim 10, further comprising: a transmitter configured to emit optical radiation; and a receiver including the pixel array and configured to detect photons of the optical radiation; wherein the processor is further configured to: generate a histogram including bins associating a number of detected photons with a given acquisition time for each pixel; and process the histogram of the pixels to provide identification of the pixels for which the target has been detected, the correctable hole pixels, and the target feature values.

14. The system of claim 10, wherein the parameters used to calculate the equivalent target detection confidence indicator include an ambient noise rate of the respective correctable hole pixel and an ambient noise rate of at least one of the neighboring pixels.

15. The system of claim 10, wherein the parameters used to calculate the equivalent target detection confidence indicator include a number of active single-photon detectors in the respective correctable hole pixel and an integration time.

16. The system of claim 10, wherein the parameters used to compute the equivalent target detection confidence indicator include a cross-talk rate of the respective correctable hole pixel and a cross-talk rate of at least one of the neighboring pixels.

17. The system of claim 10, wherein the neighboring pixels of the respective correctable hole pixel are pixels that are laterally or diagonally in contact with the respective correctable hole pixel.

18. The system of claim 10, wherein the neighborhood target detection confidence indicator is configured to trigger a binary decision: if the value of the indicator is greater than a threshold value, then a target is determined to have been detected, or if the value of the indicator is below the threshold value, then a target is determined not to have been detected.

19. A system comprising: a pixel array; a processor coupled with the pixel array; a non-transitory memory coupled to the processor and comprising instructions that, when executed by the processor, cause the processor to: identify pixels that have detected a target and correctable hole pixels that have not detected a target, the correctable hole pixels being surrounded by at least one neighboring pixel that has detected a target; compute target feature values for the pixels that have detected the target; compute, for each correctable hole pixel, an equivalent target detection confidence indicator representing a neighborhood target detection confidence indicator: given parameters including a sensitivity of a respective correctable hole pixel, detecting the target in a neighborhood of the respective correctable hole pixel would have the neighborhood target detection confidence indicator at the respective correctable hole pixel as if the target detected in the neighborhood also existed at a location of the respective correctable hole pixel, from the target feature values computed for the neighboring pixels of the correctable hole pixel; and in response to the computed equivalent target detection confidence indicator being below a detection threshold, compute a corrected target feature value associated with the respective correctable hole pixel.

20. The system of claim 19, further comprising: a transmitter configured to emit optical radiation; and a receiver comprising the pixel array and configured to detect photons of the optical radiation; wherein the memory comprises further instructions that, when executed by the processor, cause the processor to: generate a histogram comprising bins that associate a number of detected photons with a given acquisition time for each pixel; and process the histogram of the pixels to provide the identification of pixels that have detected the target, the correctable hole pixels, and the target feature values. ​

Images