Device and method for capturing a scene

Abstract

A device (100) for capturing a scene, comprising: a lighting element (110) designed to illuminate the scene with modulated light (102); and an optical sensor (120) configured to receive light (103) reflected from the scene, wherein the optical sensor (120) comprises at least one photosensitive sensor pixel (121, 121') configured as follows: charge carriers generated in the photosensitive sensor pixel (121, 121') by the reflected light (103) are stored in at least one charge storage device (123, 125) of the photosensitive sensor pixel (121, 121') during a measurement; and to selectively prevent the charge carriers from reaching the at least one charge storage (123, 125) during the measurement in order to strictly monotonically increase the sensitivity of the at least one photosensitive sensor pixel (121, 121') over the distance for reflected light (103) originating from within a measurement area of ​​the optical sensor (120).

Description

Area

[0001] The present disclosure relates to optical sensing. More particularly, examples of the present disclosure relate to an apparatus and method for sensing a scene. background

[0002] US 2021 / 028209 A1 proposes a charge collection gate with a central collection photodiode in a time-of-flight (ToF) pixel. Furthermore, US 2019 / 124278 A1 proposes image sensors with high dynamic range and noise suppression for flicker and fixed patterns. US 2017 / 212225 A1 further proposes a time-of-flight sensor with a charge compensation device.

[0003] When machine vision systems use cameras with an active illumination source (e.g., ToF cameras), nearby objects reflect more light back to the imager, while distant objects with the same reflectivity receive less light and reflect less light back. Accordingly, nearby objects can saturate the imager's pixels, while objects of the same material appear different at different distances.

[0004] In addition, the amount of electrons accumulated in the pixel depends on the object's reflectance. Determining an object's reflectance can improve object detection and recognition. Reflectance measurements can be distorted by pixel saturation caused by nearby objects.

[0005] When using an active illumination system, an image processing system also wants to receive as little light from other light sources (e.g. sunlight) as possible.

[0006] There is therefore a need for improved optical detection. Summary

[0007] The need may be satisfied by the subject matter of the appended claims.

[0008] One example relates to a device for capturing a scene. The device comprises an illumination element configured to illuminate the scene with modulated light. Additionally, the device comprises an optical sensor configured to receive the light reflected from the scene. The optical sensor comprises at least one photosensitive sensor pixel configured to store charge carriers generated in the photosensitive sensor pixel by the reflected light in at least one charge storage device of the photosensitive sensor pixel during a measurement.The at least one photosensitive sensor pixel is further configured to selectively prevent the charge carriers from reaching the at least one charge storage during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel over the distance for reflected light originating from within a measuring range of the optical sensor.

[0009] Another example relates to a method for capturing a scene. The method comprises illuminating the scene with modulated light using an illumination element. Additionally, the method comprises receiving the light reflected from the scene at an optical sensor. The optical sensor comprises at least one photosensitive sensor pixel. The method further comprises controlling the at least one photosensitive sensor pixel to store charge carriers generated in the photosensitive sensor pixel by the reflected light in at least one charge storage of the photosensitive sensor pixel during a measurement.Furthermore, the method comprises controlling the at least one photosensitive sensor pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel over the distance for reflected light originating from within a measuring range of the optical sensor. Short description of the characters

[0010] Some examples of devices and / or methods are described below solely by way of example and with reference to the accompanying figures, in which: Fig. Figure 1 illustrates an example of a scene detection device; Fig. Figure 2 illustrates an example of a photosensitive sensor element or pixel; Fig. 3 shows an example of the curve of the received light intensity and the sensitivity of a photosensitive sensor element or pixel; Fig. Figure 4 illustrates another example of a photosensitive sensor element or pixel; Fig. 5 illustrates an exemplary correlation signal along with exemplary light and state signals for a photosensitive sensor element or pixel; and Fig. 6 illustrates a flowchart of an example of a method for capturing a scene. Detailed description

[0011] Some examples will now be described in more detail with reference to the accompanying figures. However, other possible examples are not limited to the features of these detailed embodiments. Other examples may include modifications of the features, as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe specific examples is not intended to be limiting of other possible examples.

[0012] Throughout the description of the figures, identical or similar reference numerals refer to identical or similar elements and / or features, which may be implemented identically or in a modified form while providing the same or a similar function. The thickness of lines, layers, and / or regions in the figures may also be exaggerated for clarity.

[0013] When two elements A and B are combined using "or," this is to be understood as disclosing all possible combinations, i.e., only A, only B, and A and B, unless explicitly defined otherwise in the individual case. An alternative formulation for the same combinations may be "at least one of A and B" or "A and / or B." This applies equivalently to combinations of more than two elements.

[0014] If a singular form is used, such as "a," "an," and "the," and the use of only a single element is neither explicitly nor implicitly defined as mandatory, further examples may also use multiple elements to implement the same function. If a function is subsequently described as being implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity.It is further understood that the terms "comprises," "comprising," "has," and / or "having," when used, describe the presence of the specified features, integers, steps, operations, processes, elements, components, and / or a group thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components, and / or a group thereof.

[0015] Fig. 1 illustrates an example of a device 100 for capturing a scene.

[0016] The device comprises an illumination element (circuitry, device) 110 configured to illuminate the scene with modulated light 102 (e.g., modulated light pulses). The illumination element 110 may comprise any number of light sources. The illumination element 110 may, for example, comprise one or more light-emitting diodes (LEDs) and / or one or more laser diodes (e.g., one or more vertical-cavity surface-emitting lasers (VCSELs)) that are triggered based on one or more illumination signals.

[0017] An object 101 is positioned in the scene and reflects the emitted, modulated light 102. The object 101 can be any type of physical object. The reflected light 203 travels at least partially back to the device 100.

[0018] The device 100 comprises an optical sensor 120 configured to receive (detect) the light 103 reflected from the scene. The optical sensor 120 may comprise various components, such as optics (e.g., one or more lenses) and electronic circuitry. In particular, the electronic circuitry comprises at least one photosensitive sensor element or pixel (e.g., a photonic mixer device, PMD, or a charge-coupled device, CCD). The optical sensor 120 may, for example, comprise a plurality of photosensitive sensor elements or pixels (i.e., N ≥ 2 photosensitive sensor elements or pixels). The at least one photosensitive sensor element or pixel may be controlled based on one or more control (reference) signals.

[0019] In general, the optical sensor 120 can be any type of sensor capable of measuring a physical quantity of light (e.g., the reflected light 203) and then translating it into a form that can be read by further electronic circuitry. In particular, the optical sensor 120 can be any type of sensor capable of converting light or a change in light into an electronic signal. The optical sensor 120 can be, for example, a frame-based image sensor comprising photosensitive sensor elements or pixels that operate uniformly and synchronously to detect light frame by frame. Alternatively, the optical sensor 120 can be, for example, a ToF sensor that measures a ToF of the emitted modulated light 102 using the reflected light 103 received at the optical sensor 120. Still alternatively, the optical sensor 120 can be, for example, an event-based image processing sensor, such asA dynamic vision sensor (also known as an event camera, neuromorphic camera, or silicon retina) that responds to local changes in brightness. An event-based vision sensor does not capture images / frames using a shutter like a conventional image sensor. Instead, the photosensitive sensing element, or pixels, of an event-based vision sensor operate independently and asynchronously, meaning they detect changes in brightness when they occur and remain still otherwise. An event-based vision sensor can offer high temporal resolution, high dynamic range, under- / overexposure, and motion blur compared to frame-based image sensors.

[0020] An exemplary photosensitive sensor element or pixel 121 is shown in Fig. 2. If the optical sensor 120 comprises a plurality of photosensitive sensor elements or pixels, one or more of the plurality of photosensitive sensor elements or pixels may be formed and operated as the photosensitive sensor element or pixel 121 described below.

[0021] The photosensitive sensor element or pixel 121 comprises a semiconductor material / substrate 122 (e.g., silicon). The reflected light 103 penetrates the semiconductor material 122 and causes the generation of charge carriers (e.g., electrons or holes) in the semiconductor material 122.

[0022] The photosensitive sensor element or pixel 121 also comprises at least one charge storage device such as that shown in Fig. 2 shown charge storage 123. Although in Fig. 2 illustrates exactly one charge storage device for the photosensitive sensor element or pixel 121, it should be noted that the present disclosure is not limited thereto. In general, any number M ≥ 1 of charge storage devices per photosensitive sensor element or pixel may be used. The at least one charge storage device may, for example, be a (corresponding) capacitor or a potential well formed in the semiconductor material 122 of the photosensitive sensor element or pixel 121. It should be noted that the charge storage device 123 in Fig. 1 is shown as a separate element for illustrative purposes only. The at least one charge storage device is part of the photosensitive sensor element or pixel 121. The at least one charge storage device enables the (e.g., selective) storage of the generated charge carriers.

[0023] During a measurement (e.g., a single exposure of the optical sensor 120), the at least one photosensitive sensor element or pixel 121 is configured (e.g., controlled by the optical sensor 120) to store (e.g., selectively) the charge carriers generated during the measurement by the reflected light 103 in the semiconductor material 122 of the photosensitive sensor pixel 121 in the at least one charge storage device 123. The output (value, data) of the at least one photosensitive sensor element or pixel 121 for the measurement is determined based on the charge state(s) of the at least one charge storage device 123.

[0024] In addition, the photosensitive sensor element or pixel 121 includes a drain terminal 124. The drain terminal 124 is a terminal that allows the generated charge carriers to be selectively drained from the photosensitive sensor element or pixel 121.

[0025] During the measurement, the at least one photosensitive sensor element or pixel 121 is further configured (e.g., controlled by the optical sensor 120) to selectively dissipate the generated charge carriers in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel 121 to reflected light 103 originating within a (target) measurement range of the optical sensor 120 over the distance. In other words, the at least one photosensitive sensor pixel 121 is further configured to selectively prevent the charge carriers from reaching the at least one charge storage 123 during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel 121 over the distance to reflected light 103 originating from within a measurement range of the optical sensor 120.

[0026] The (target) measurement range of the optical sensor 120 is a distance range in which a respective quantity of one or more objects, such as the object 101, is to be measured by the optical sensor 120. The sensitivity of the at least one photosensitive sensor pixel 121 refers to the extent to which the at least one photosensitive sensor pixel 121 reacts to the reflected light 103 originating from a specific distance. The higher the sensitivity of the at least one photosensitive sensor pixel 121 to reflected light 103 from a specific distance, the higher the output (value, data) of the at least one photosensitive sensor pixel 121 for reflected light 103 from this distance.

[0027] Only the charge carriers stored in the at least one charge storage device 123 during the measurement contribute to the output (value, data) of the at least one photosensitive sensor element or pixel 121. Accordingly, the distance-dependent sensitivity of the optical sensor 120 can be adjusted (shaped) by selectively preventing some (a portion) of the generated charge carriers from reaching the at least one charge storage device 123 during the measurement.

[0028] In the example of Fig. 2, the photosensitive sensor element or pixel 121 includes the drain terminal 124 to drain a portion of the generated charge carriers and thus prevent them from reaching the at least one charge storage device 123. However, it should be noted that the present disclosure is not so limited. In other examples, the drain terminal 124 may be omitted, and the photosensitive sensor element or pixel 121 may be configured to electrically decouple (isolate) the at least one charge storage device 123 from the semiconductor material 122 so that the generated charge carriers cannot reach the at least one charge storage device 123. In general, any technique may be used to selectively prevent a portion of the charge carriers generated during the measurement from reaching the at least one charge storage device 123.

[0029] For example, the at least one photosensitive sensor pixel 121 can be configured to selectively prevent the charge carriers from reaching the at least one charge storage device 123 based on the at least one drive signal for the photosensitive sensor pixel 121. Similarly, the at least one photosensitive sensor pixel 121 can be configured to store the charge carriers (e.g., selectively) in the at least one charge storage device 123 based on the at least one drive signal for the photosensitive sensor pixel 121. The at least one drive signal can, for example, be received by the at least one photosensitive sensor pixel 121 from another circuit arrangement of the optical sensor 120 or from circuit arrangement external to the optical sensor 120.

[0030] The at least one photosensitive sensor pixel 121 therefore effectively comprises a "shutter" such as the drain terminal 124, which makes it possible to prevent generated charge carriers from reaching the at least one charge storage device 123. As described above, the "shutter function" of the at least one photosensitive sensor pixel 121 can be controlled based on the one or more control signals. Thus, it is possible to control whether or not the generated charge carriers are accumulated during the measurement.

[0031] For example, the at least one photosensitive sensor element or pixel 121 can provide an integration state during which the charge carriers generated by the reflected light 103 during the measurement are selectively stored in the at least one charge storage device 123. Furthermore, the at least one photosensitive sensor element or pixel 121 can provide a bypass (non-integrating) state during which the charge carriers generated by the reflected light 103 during the ToF measurement are prevented from reaching the at least one charge storage device 123 (e.g., the charge carriers generated by the reflected light 103 during the ToF measurement can be bypassed via the bypass terminal 124, or the at least one charge storage device 123 can be electrically decoupled / isolated from the semiconductor material 122 during the bypass / non-integrating state).During the measurement, the at least one photosensitive sensor element or pixel 121 can operate according to a state pattern. The state pattern can, for example, alternately comprise the integration state and the discharge state, such that each two consecutive integration states are separated by a discharge state. By alternately discharging and storing the charge carriers over time and by selecting (setting, adjusting) suitable durations for the one or more integration states and the one or more discharge states, the one or more times and durations of the one or more discharge states can be adjusted during the measurement such that the sensitivity of the at least one photosensitive sensor pixel 121 increases strictly monotonically over the distance within the measuring range of the ToF sensor 100. The state pattern can, for example,into which one or more control signals for the photosensitive sensor pixel 121 are encoded.

[0032] Optionally, the photosensitive sensor element or pixel 121 may comprise one or more further elements such as control gates for controlling the flow of the generated charge carriers within the photosensitive sensor element or pixel 121 (e.g., two or more modulation gates and / or one or more diverter gates) and / or one or more readout terminals for reading the at least one charge storage device.

[0033] Due to the increasing sensitivity of the at least one photosensitive sensor element or pixel 121 within the measurement range of the optical sensor 120, the measurement is less sensitive to reflected light 103 from the immediate vicinity of the optical sensor 120. In other words, the sensitivity of the at least one photosensitive sensor element or pixel 121 is shaped (adjusted) such that a higher sensitivity is provided for distances (regions) that are farther away from the optical sensor 120. Consequently, close distances (regions) receive a lower sensitivity, and far distances (regions) receive a higher sensitivity. Furthermore, saturation of the at least one photosensitive sensor element or pixel 121 by reflected light 103 from the immediate vicinity of the optical sensor 120 can be avoided. This can prevent distortion of the measurement due to pixel saturation.

[0034] Fig. 3 shows an exemplary curve 310 of the sensitivity of the photosensitive sensor element or pixel 121. Furthermore, Fig. 3 an exemplary curve 320 of the light intensity (intensity) of the reflected light 103 received at the optical sensor 120. The abscissa in Fig. 3 indicates the distance between the optical sensor 120 and the object 101. The ordinate indicates the respective value of the sensitivity of the photosensitive sensor element or pixel 121 and the luminous intensity of the reflected light 103. In the example of Fig. 3, it is assumed that the distances of the entire abscissa are within the measuring range of the optical sensor 120.

[0035] As can be seen from the exemplary curve 310, the sensitivity of the photosensitive sensor element or pixel 121 increases strictly monotonically with the distance (ie the sensitivity of the photosensitive sensor element or pixel 121 increases strictly monotonically with larger distances between the optical sensor 120 and the object 101). In the example of Fig. 3, the sensitivity of the photosensitive sensor element or pixel 121 increases essentially quadratically with the distance within the measurement range. In other words, the sensitivity S of the photosensitive sensor element or pixel 121 can be set, for example, as follows: S(d)∝d2 where d indicates the distance between the optical sensor 120 and the object 101 that reflects the emitted modulated light 102 back to the optical sensor 120.

[0036] To increase sensitivity over distance, the at least one photosensitive sensor element or pixel 121 can be controlled such that the proportion of discharged charge carriers decreases with increasing distance of the optical sensor 120 from the object 101 in the scene (which causes the reflected light). In other words, the at least one photosensitive sensor element or pixel 121 can be controlled such that the proportion of charge carriers prevented from reaching the at least one charge storage device 123 is reduced. Accordingly, the at least one photosensitive sensor element or pixel 121 can be configured such that the proportion of charge carriers stored in the at least one charge storage device 121 increases with increasing distance of the optical sensor 120 from the object 101 in the scene (which causes the reflected light).

[0037] As can be seen from curve 320, the luminous intensity of the reflected light 203 decreases with the distance between the optical sensor 120 and the object 101. For example, it can be assumed (e.g., with point-like light sources) that the luminous intensity decreases according to the inverse square law. This means that the distance-dependent luminous intensity of the reflected light 103 received at the optical sensor 120 can be assumed as follows: I(d)∝1d2 where I indicates the luminous intensity of the reflected light 103 received at the optical sensor 120.

[0038] Since the sensitivity of the optical sensor 120 to light from the near area of ​​the optical sensor 120 is reduced by the diversion, saturation of the at least one photosensitive sensor element or pixel 121 due to strong reflections from the near area of ​​the optical sensor 120 can be avoided. In addition, glare or scattered light effects caused by reflections of the emitted modulated light 102 by an object in the immediate vicinity of the optical sensor 120 can be eliminated or at least reduced.

[0039] The output of the at least one photosensitive sensor element or pixel 121 for the measurement scales with the luminous intensity of the reflected light 103 received from the object 101. For example, an output value output by the at least one photosensitive sensor element or pixel 121 for the measurement can be determined by the product of the luminous intensity of the reflections received from the object 101 during that measurement and the sensitivity of the photosensitive sensor element or pixel 121 at the distance of the object 101 that caused the reflected light 103.

[0040] Accordingly, the sensitivity of the photosensitive sensor element or pixel 121 can be adjusted such that the sensitivity for the measurement increases strictly monotonically over the distance within the measurement range of the optical sensor 120, in order to compensate for the decreasing light intensity of the reflected light 203 received at the optical sensor 120. The decreasing light intensity of the reflected light 203 received at the optical sensor 120 can be compensated for, in particular, if the sensitivity of the photosensitive sensor element or pixel 121 increases substantially quadratically over the distance within the measurement range. Consequently, when the object 101 is within the measurement range of the optical sensor 120, an output value of the photosensitive sensor element or pixel 121 or the optical sensor 120 for the measurement can be substantially independent of the distance between the optical sensor 120 and the object 101.

[0041] The output value of the at least one photosensitive sensor pixel 121 for the measurement is proportional to the reflectivity of the object 101, since the reflectivity of the object 101 determines how much light hits the optical sensor 120 and thus the at least one photosensitive sensor pixel 121 during the measurement. The reflectivity of an object determines the ratio between the amount of emitted, modulated light 102 arriving at a surface of the object 101 and the amount of light reflected back to the optical sensor 120 (i.e., the amount of reflected light 103). If the object 101 is within the measurement range of the optical sensor 120, the output value of the at least one photosensitive sensor pixel 121 therefore varies with the reflectivity of the object 101—regardless of the distance between the optical sensor 120 and the object 101.Therefore, the output value of the at least one photosensitive sensor pixel 121 may make it possible to characterize the reflectivity of the object 101 if the sensitivity of the at least one photosensitive sensor pixel 121 is designed as described above.

[0042] To determine the reflectivity of the object 101, the Fig. 1 may optionally include processing circuitry 130 coupled to the optical sensor 120. The processing circuitry 130 may be, for example, a single dedicated processor, a single shared processor, or a plurality of individual processors, some or all of which may be shared, digital signal processor (DSP) hardware, an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA). The processing circuitry 130 may optionally be coupled, for example, to a read-only memory (ROM) for storing software, a random access memory (RAM), and / or a non-volatile memory.The processing circuitry 130 is configured to determine a reflectivity value indicative of a reflectivity of the object 101 in the scene based on the output value of the at least one photosensitive sensor pixel 121 for the measurement.

[0043] For example, processing circuitry 130 may determine the reflectivity value by applying at least one correction to the output value of the at least one photosensitive sensor pixel 121 for the measurement. The output value of the at least one photosensitive sensor pixel 121 for the measurement may, for example, be scaled and / or offset-corrected to obtain the reflectivity value. Accordingly, systematic errors (e.g., noise) may be corrected.

[0044] For example, the processing circuitry 130 may output additional data indicative of the reflectivity value (e.g., a two-dimensional image).

[0045] In some cases, the processing circuitry 130 may also be coupled to the lighting element 110 and configured to control the lighting element 110 as described above.

[0046] The device 100 may include additional hardware - conventional and / or custom.

[0047] The following description focuses on an example of the present disclosure in which the optical sensor 120 is a ToF sensor. Accordingly, the measurement is a ToF measurement.

[0048] Fig. 4 shows an exemplary photosensitive sensor element or pixel 121' for the ToF sensor 120. The ToF sensor 120 may include at least one photosensitive sensor element or pixel implemented like the photosensitive sensor element or pixel 121'. For example, the ToF sensor 120 may include a plurality of photosensitive sensor elements or pixels implemented like the photosensitive sensor element or pixel 121'.

[0049] The photosensitive sensor element or pixel 121' is substantially identical to the photosensitive sensor element or pixel 121 described above with respect to Fig. 3. Compared to the photosensitive sensor element or pixel 121, the photosensitive sensor element or pixel 121' comprises an additional charge storage 125. The additional charge storage 125 can be implemented as described above for the charge storage 123. It should be noted that the charge storages 123 and 125 in Fig. 5 are shown as separate elements for illustrative purposes only. The two charge storage devices are part of the photosensitive sensor element or pixel 121'. The two charge storage devices 123 and 125 enable the selective storage of the generated charge carriers. Although in Fig. While exactly two charge storage devices are shown for the photosensitive sensor element or pixel 121' in FIG. 5, it should be noted that the present disclosure is not limited thereto. In general, any number K ≥ 2 of charge storage devices may be used per photosensitive sensor element or pixel of the ToF sensor 120.

[0050] Similar to the example described above for the photosensitive sensor element or pixel 121 with the single charge storage 123, the at least one photosensitive sensor pixel 121' is configured such that the generated charge carriers are selectively stored in the at least two charge storages 123 and 125 of the at least one photosensitive sensor element or pixel 121' during the ToF measurement. The output (value, data) of the at least one photosensitive sensor element or pixel 121 for the ToF measurement is determined based on the charge levels of the at least two charge storages 123 and 125. For example, the output value of the at least one photosensitive sensor element or pixel 121' for the ToF measurement can be based on the difference between the charge levels of the two charge storages of the at least two charge storages 123 and 125.In other words, the output value of the at least one photosensitive sensor element or pixel 121' for the ToF measurement can be based on a charge difference between the stored charges in the at least two charge storage devices 123 and 125. Background light impinging on the ToF sensor 120 causes the generation of unwanted charge carriers. The charge carriers caused by the background light are stored in all of the at least two charge storage devices 123 and 125. If the output value of the at least one photosensitive sensor element or pixel 121' for the ToF measurement is based on a charge difference between the stored charges in the at least two charge storage devices 123 and 125, the effect of the background light on the ToF measurement can be suppressed or at least mitigated.For example, the at least one photosensitive sensor pixel 121' can be configured to store the charge carriers generated during the ToF measurement in each of the at least two charge storage devices 123 and 125 for the same amount of time. Accordingly, the amount of charge carriers caused by the background light is substantially equal in all of the at least two charge storage devices 123 and 125, so that they essentially cancel each other out. This allows the measurement of reflectivity with the ToF sensor 120 to be further improved.

[0051] Analogous to what has been described above, the at least one photosensitive sensor pixel 121' is configured such that it selectively prevents the generated charge carriers from reaching the at least two charge storage devices 123 and 125 during the ToF measurement. For example, the at least one photosensitive sensor pixel 121' can be configured such that it selectively prevents the charge carriers from reaching the at least one charge storage device 123 and 125 during the ToF measurement by selectively discharging the generated charge carriers via the discharge terminal 124. However, it should be noted that the present disclosure is not so limited. In other examples, the discharge terminal 124 can be omitted and the photosensitive sensor element or pixel 121 can, for example,be configured to electrically decouple (isolate) the at least two charge storage devices 223 and 224 from the semiconductor material 122 such that the generated charge carriers cannot reach the at least two charge storage devices 223 and 224. In general, any technique can be used to selectively prevent a portion of the charge carriers generated during the measurement from reaching the at least two charge storage devices 123 and 125.

[0052] Like the at least one photosensitive sensor pixel 121 described above, the at least one photosensitive sensor pixel 121' may allow selectively preventing some (a portion) of the generated charge carriers from reaching the at least two charge storages 123 and 125 during the ToF measurement. Since during the ToF measurement only the charge carriers stored in the at least two charge storages 123 and 125 contribute to the output (value, data) of the at least one photosensitive sensor element or pixel 121 for the ToF measurement, the distance-dependent sensitivity of the ToF sensor 120 can be adjusted (shaped).

[0053] The sensitivity of a photosensitive sensor element or pixel for a ToF measurement is given by its (light intensity-independent) correlation function for the ToF measurement. The (light intensity-independent) correlation function of the at least one photosensitive sensor element or pixel 121' for a ToF measurement indicates the distance-dependent correlation of the reflected light 103 of the at least one photosensitive sensor pixel 121 with one or more control signals used to control the at least one photosensitive sensor element or pixel 121' during the ToF measurement, without taking into account (i.e., ignoring, disregarding) an intensity of the reflected light 103.In other words, the (light intensity-independent) correlation function only describes the distance dependence of the output (value, data) of the photosensitive sensor pixel, but not the dependence of the output (value, data) of the photosensitive sensor pixel on the intensity of the reflected light 103. Analogous to the sensitivity described above, the correlation function of the at least one photosensitive sensor pixel 121' for the ToF measurement increases strictly monotonically over the distance within the (target) measuring range of the ToF sensor 120. The correlation function can, for example, increase essentially quadratically over the distance within the measuring range.

[0054] Fig. 5 shows in sub-figure (a) an exemplary correlation function 510 of the at least one photosensitive sensor element or pixel 121' for a ToF measurement. The abscissa of sub-figure (a) indicates the distance between the ToF sensor 120 and the object 101. The ordinate indicates the value of the correlation function. In the example of Fig. 5, the (target) measuring range 515 of the ToF sensor 120 ranges from approximately 0 to 1.25. Similar to the Fig. 3, the correlation function 510 increases strictly monotonically over the distance within the measuring range 515. In particular, the correlation function 510 increases essentially quadratically over the distance within the measuring range 515. The correlation function 510 can only be valid within the measuring range 515.

[0055] Partial figure (b) of Fig. Figure 5 shows an exemplary sequence of modulated light pulses 520 emitted into the scene by the light-emitting element 120 for the ToF measurement. The abscissa of sub-figure (b) indicates time. The ordinate indicates the amplitude of the illumination signal that controls the light-emitting element 120 to emit the light pulses.

[0056] Partial figure (c) of Fig. 5 shows an exemplary temporal progression 530 of the state of the at least one photosensitive sensor element or pixel 121' during the ToF measurement. The abscissa of sub-figure (c) indicates time. The ordinate indicates the state of the at least one photosensitive sensor element or pixel 121. As in sub-figure (c) of Fig. As shown in Figure 5, two states are possible.

[0057] While the at least one photosensitive sensor element or pixel 121' is in the "pixel storage" state, the at least one photosensitive sensor element or pixel 121' selectively stores the charge carriers generated by the reflected light 103 during the ToF measurement in the at least two charge storage devices 123 and 125. For example, the at least one photosensitive sensor element or pixel 121' can selectively store the generated charge carriers in the at least two charge storage devices 123 and 125 according to a modulation code encoded in the one or more drive signals used to drive the at least one photosensitive sensor element or pixel 121' during the ToF measurement.In other words, the at least one photosensitive sensor element or pixel 121 can provide an integration state during which the charge carriers generated by the reflected light 103 during the ToF measurement are selectively stored in the at least two charge storage devices 123 and 125.

[0058] While the at least one photosensitive sensor element or pixel 121' is in the "drain" state, the at least one photosensitive sensor element or pixel 121' selectively drains the charge carriers generated by the reflected light 103 during the ToF measurement via the drain connection 124. While the at least one photosensitive sensor element or pixel 121' is in the "drain" state, no charge carriers are stored in the charge storage devices 123 and 125. In other words, the at least one photosensitive sensor element or pixel 121 provides a drain (non-integrating) state during which the charge carriers generated by the reflected light 103 during the ToF measurement are prevented from reaching the at least one charge storage device 123. The charge carriers generated by the reflected light 103 during the ToF measurement cannot be drained solely via the drain connection 124.In other examples, the at least one charge storage 123 may be electrically decoupled (isolated) from the semiconductor material 122 during the dissipation (non-integrating) state.

[0059] The state of the at least one photosensitive sensor element or pixel 121' is controlled by the one or more control signals used to control the at least one photosensitive sensor element or pixel 121' during the ToF measurement. For example, the at least one photosensitive sensor element or pixel 121' can operate during the ToF measurement according to a state pattern encoded in the one or more control signals. As shown in sub-figure (c), the state pattern can, for example, alternately comprise the integration state and the derivation state, such that each two consecutive integration states are separated by a derivation state.By alternately discharging and storing the charge carriers over time and by selecting (adapting) suitable durations for the one or more integration states and the one or more discharging states, the one or more times and durations of the one or more discharging states during the ToF measurement can be adjusted such that the correlation function 510 of the at least one photosensitive sensor pixel 121' for the ToF measurement increases strictly monotonically over the distance within the measuring range 515 of the ToF sensor.

[0060] Partial figure (d) of Fig. Figure 5 shows an exemplary distance-dependent fraction 540 of the generated charge carriers that are selectively diverted via the diverting terminal 124. The abscissa of sub-figure (d) indicates the distance between the ToF sensor 120 and the object 101. The ordinate indicates the fraction of diverted charge carriers. As can be seen from sub-figure (d), the at least one photosensitive sensor element or pixel 121 is controlled to reduce the fraction of diverted charge carriers with increasing distance of the object 101 (which causes the incident light 103) from the ToF sensor 120. In other words, the at least one photosensitive sensor element or pixel 121 is controlled to increase a proportion of the generated charge carriers that are selectively stored in the at least two charge storage devices 123 and 125 with increasing distance of the ToF sensor 120 from the object 101 that causes the incident light 103.The reduction of the fraction of discharged charge carriers over the distance is achieved by a suitable adjustment of the duration for the one or more integration states and the one or more discharge states in the state pattern.

[0061] As can be seen from sub-figure (d), only a portion of the charge carriers generated by reflected light 103 coming from the beginning of the measuring area 515 are stored in the at least two charge storage devices 123 and 125. The stored charge carriers contribute to the correlation function 510 illustrated in sub-figure (a). The remainder of the charge carriers are removed (drained) from the at least one photosensitive sensor element or pixel 121' via the diverting terminal 124 and are thus not stored in the at least two charge storage devices 123 and 125. This is achieved by the fact that the reflected light 103 causing these charge carriers arrives while the at least one photosensitive sensor element or pixel 121' is in the "draining" state.Charge carriers caused by reflected light 103 from the far end of the measurement range are essentially completely stored in the at least two charge storage devices 123 and 125 and therefore contribute more strongly to the correlation function 510 shown in sub-figure (a). Accordingly, the amplitude of the correlation function 510 increases strictly monotonically over the distance within the measurement range 515 of the ToF sensor 120.

[0062] There are various, essentially infinite possibilities as to how the one or more control signals that control the at least one photosensitive sensor element or pixel 121' of the ToF sensor 120 and the one or more illumination signals for controlling the illumination element 110 can be formed (shaped) to generate correlation functions that, as in the example described above, Fig. 5. Therefore, it should be noted that the present disclosure is not limited to the specific configuration (shape) of the one or more drive signals and the one or more illumination signals. Fig. The signal structures shown in Figure 5 are therefore for illustrative purposes only. For example, the signal structures shown in sub-figure (c) of Fig. 5 can be repeated several times or mixed. Alternatively, other signal patterns can be used. Similarly, the length of the light pulses shown in sub-figure (c) of Fig. 5, may be uniform or have a different pattern than in part figure (c) of Fig. 5. In general, the one or more drive signals and the one or more illumination signals are chosen such that charge carriers caused by incident light from closer areas are dissipated to some extent, causing the respective correlation function to increase strictly monotonically with distance. For example, the emitted light pulses 520 and the one or more drive signals (which can be understood as pixel state signal(s)) can be shaped to approximate the inverse of the light dilution function (see mathematical expression (2)). The frequency of the one or more drive signals and the illumination signal can be adapted to the desired measurement range, since the distant end of the measurement scales linearly with frequency.

[0063] The depth sensing can be used to adapt the measurement range of the ToF sensor 120 to the distance of the object 101. Adapting the measurement range to the distance of the object can make it possible to increase the energy efficiency and / or energy consumption of the device 100, since no active light, i.e., emitted modulated light 102, is unnecessarily wasted or diverted. For example, the ToF sensor 120 can be configured to perform one or more second ToF measurements before performing the ToF measurement described above for the reflectivity measurement. The processing circuitry 130 can be configured accordingly to determine a distance value indicative of the distance to the object 101 based on an output of the ToF sensor 120 for the one or more second ToF measurements. The one or more second ToF measurements and the determination of the distance value can be performed according to conventional ToF depth measurement principles (e.g.,using a corresponding oscillating correlation function). Further, the processing circuitry 130 may be configured to adjust the measurement range of the ToF sensor 120 for the above-described ToF measurement to include the object 101 based on the distance value. For example, the processing circuitry 130 may adjust the frequency and / or another parameter of the one or more drive signals and the one or more illumination signals based on the determined distance value. Optionally, the processing circuitry 130 may output further data indicative of the distance value. For example, one or more images indicative of the reflectivity value and / or the distance value may be output by the processing circuitry 130.

[0064] To summarize the proposed optical detection, Fig.6 shows a flowchart of an example of a method 600 for capturing a scene. The method 600 includes illuminating 602 the scene with modulated light using an illumination element. Additionally, the method 600 includes receiving 604 the light reflected from the scene at an optical sensor. The ToF sensor includes at least one photosensitive sensor element or pixel. The method 600 further includes controlling 606 the at least one photosensitive sensor element or pixel to store charge carriers generated in the photosensitive sensor element or pixel by the reflected light in at least one charge storage of the photosensitive sensor element or pixel during a measurement.Furthermore, the method 600 comprises controlling 608 the at least one photosensitive sensor element or pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the measurement, 600 in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor element or pixel over the distance for reflected light that arises within a measuring range of the optical sensor.

[0065] Method 600 can make it possible to avoid saturation of the at least one photosensitive sensor element or pixel by reflected light from the immediate vicinity of the optical sensor. This can prevent measurement distortion due to pixel saturation.

[0066] Further details and aspects of the method 600 are explained in connection with the proposed technique or one or more examples described above. The method 600 may include one or more additional optional features corresponding to one or more aspects of the proposed technique or one or more of the examples described above. Some further features of the method 600 are described below by way of example, although it should be noted that the method 600 is not limited thereto.

[0067] For example, as described above, the sensitivity can increase quadratically over the distance within the measuring range.

[0068] The method 600 may further optionally include determining a reflectivity value indicative of a reflectivity of an object in the scene based on an output value of the at least one photosensitive sensor pixel for the measurement.

[0069] The optical sensor may, for example, be a ToF sensor in method 600. Accordingly, the measurement may be a ToF measurement.

[0070] Furthermore, method 600 may include performing one or more second ToF measurements with the ToF sensor prior to performing the ToF measurement. A distance value indicative of a distance to the object may optionally be determined in method 600 based on an output of the ToF sensor for the one or more second ToF measurements. Further, method 600 may include adjusting the measurement range of the ToF sensor for the ToF measurement based on the distance value to include the object.

[0071] The control 608 of the at least one photosensitive sensor element or pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the ToF measurement may, for example, comprise controlling the at least one photosensitive sensor element or pixel to selectively divert the charge carriers.

[0072] The examples described herein can be summarized as follows: Examples relate to a device for capturing a scene. The device comprises an illumination element configured to illuminate the scene with modulated light. Additionally, the device comprises an optical sensor configured to receive the light reflected from the scene. The optical sensor comprises at least one photosensitive sensor pixel configured to store charge carriers generated in the photosensitive sensor pixel by the reflected light in at least one charge storage device of the photosensitive sensor pixel during a measurement.The at least one photosensitive sensor pixel is further configured to selectively prevent the charge carriers from reaching the at least one charge storage during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel over the distance for reflected light originating from within a measuring range of the optical sensor.

[0073] In some examples, the sensitivity increases quadratically with the distance within the measuring range.

[0074] According to some examples, the photosensitive sensor pixel is further configured such that a proportion of the charge carriers stored in the at least one charge storage increases with increasing distance of the optical sensor from an object in the scene that causes the reflected light.

[0075] In some examples, the at least one photosensitive sensor pixel is further configured to selectively prevent the charge carriers from reaching the at least one charge storage based on at least one received drive signal.

[0076] According to some examples, the apparatus further comprises processing circuitry configured to determine a reflectivity value indicative of a reflectivity of an object in the scene based on an output value of the at least one photosensitive sensor pixel for the measurement.

[0077] According to some examples, wherein the at least one photosensitive sensor pixel provides an integration state during which the charge carriers are stored in the at least one charge storage device, wherein the at least one photosensitive sensor pixel further provides a non-integrating state during which the charge carriers are prevented from reaching the at least one charge storage device, wherein the at least one photosensitive sensor pixel is controlled to operate according to a state pattern during measurement, wherein the state pattern alternately comprises the integration state and the non-integrating state.

[0078] In some examples, the optical sensor is a ToF sensor, where the measurement is a ToF measurement.

[0079] According to some examples, a correlation function of the at least one photosensitive sensor pixel for the ToF measurement increases strictly monotonically over the distance within the measurement range of the ToF sensor.

[0080] In some examples, the correlation function indicates the distance-dependent correlation of the at least one photosensitive sensor pixel of the reflected light with one or more control signals and without taking into account an intensity of the reflected light, wherein the at least one photosensitive sensor pixel is controlled based on the one or more control signals.

[0081] In some examples, the correlation function increases quadratically with the distance within the measurement range.

[0082] In some examples, the at least one photosensitive sensor pixel is configured to: selectively store the charge carriers in at least two charge storages of the photosensitive sensor pixel during the ToF measurement; and to selectively prevent the charge carriers from reaching the at least two charge storages during the ToF measurement.

[0083] According to some examples, an output value of the at least one photosensitive sensor pixel for the ToF measurement is based on a charge difference between stored charges in the at least two charge storages.

[0084] According to some examples, the at least one photosensitive sensor pixel provides an integration state during which the charge carriers are stored in the at least two charge storage devices, wherein the at least one photosensitive sensor pixel further provides a non-integrating state during which the charge carriers are prevented from reaching the at least two charge storage devices, wherein the at least one photosensitive sensor pixel is controlled to operate according to a state pattern during the ToF measurement, wherein the state pattern alternately comprises the integration state and the non-integrating state.

[0085] In some examples, the ToF sensor is further configured to perform one or more second ToF measurements before performing the ToF measurement, and wherein the processing circuitry is configured to: determine a distance value indicative of a distance to the object based on an output of the ToF sensor for the one or more further ToF measurements; and adjust, based on the distance value, the measurement range of the ToF sensor for the ToF measurement to include the object.

[0086] According to some examples, the at least one photosensitive sensor pixel is configured to selectively prevent the charge carriers from reaching the at least one charge storage during the ToF measurement by selectively diverting the charge carriers.

[0087] In alternative examples, the optical sensor is an image sensor or a dynamic vision sensor.

[0088] Other examples relate to a method for capturing a scene. The method comprises illuminating the scene with modulated light using an illumination element. Additionally, the method comprises receiving the light reflected from the scene at an optical sensor. The optical sensor comprises at least one photosensitive sensor pixel. The method further comprises controlling the at least one photosensitive sensor pixel to store charge carriers generated in the photosensitive sensor pixel by the reflected light in at least one charge storage of the photosensitive sensor pixel during a measurement.Furthermore, the method comprises controlling the at least one photosensitive sensor pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel over the distance for reflected light originating from within a measuring range of the optical sensor.

[0089] In some examples, the sensitivity increases quadratically with the distance within the measuring range.

[0090] According to some examples, the method further comprises determining a reflectivity value indicative of a reflectivity of an object in the scene based on an output value of the at least one photosensitive sensor pixel for the measurement.

[0091] In some examples, the optical sensor is a ToF sensor and the measurement is a ToF measurement.

[0092] According to some examples, the method further comprises: performing one or more second ToF measurements with the ToF sensor prior to performing the ToF measurement; determining a distance value indicative of a distance to the object based on an output of the ToF sensor for the one or more further ToF measurements; and adjusting, based on the distance value, the measurement range of the ToF sensor for the ToF measurement to include the object.

[0093] In some examples, controlling the at least one photosensitive sensor pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the ToF measurement includes controlling the at least one photosensitive sensor pixel to selectively drain the charge carriers.

[0094] Examples of the present disclosure may provide a camera system with active illumination (e.g., ToF) that increases its sensitivity over distance. Examples of the present disclosure may, for example, enable the improvement of ToF sensor-based facial recognition in smartphones or object and material recognition in robotics and automation. Some examples of the present disclosure may enable reflectivity measurements using the sensor's leakage gate feature.

[0095] The aspects and features described in connection with a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the feature into the further example.

[0096] It is further understood that the disclosure of multiple steps, processes, operations, or functions disclosed in the specification or claims should not be construed as necessarily being in the described order, unless explicitly stated in the individual case or absolutely necessary for technical reasons. Therefore, the foregoing description does not limit the performance of multiple steps or functions to any particular order. Furthermore, in further examples, a single step, function, process, or operation may include and / or be broken down into multiple sub-steps, functions, processes, or operations.

[0097] If some aspects are described in the context of a device or system, these aspects are also to be understood as a description of the corresponding method. For example, a block, device, or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in the context of a method are also to be understood as a description of a corresponding block, element, property, or functional feature of a corresponding device or system.

[0098] The following claims are hereby incorporated into the Detailed Description, and each claim may stand on its own as a separate example. It should also be noted that although a dependent claim in the claims refers to a particular combination with one or more other claims, other examples may include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly contemplated unless it is specifically stated that a particular combination is not intended. Furthermore, features of a claim for any other independent claim are also intended to be included, even if that claim is not directly defined as dependent on that other independent claim.

Claims

[1] A device (100) for capturing a scene, comprising: a lighting element (110) configured to illuminate the scene with modulated light (102); and an optical sensor (120) configured to receive light (103) reflected from the scene, wherein the optical sensor (120) comprises at least one photosensitive sensor pixel (121, 121') configured: storing charge carriers generated in the photosensitive sensor pixel (121, 121') by the reflected light (103) in at least one charge storage device (123, 125) of the photosensitive sensor pixel (121, 121') during a measurement; and selectively prevent the charge carriers from reaching the at least one charge storage (123, 125) during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel (121, 121') over the distance for reflected light (103) originating from within a measuring range of the optical sensor (120). [2] The device according to claim 1, wherein the sensitivity increases quadratically with the distance within the measuring range. [3] The device according to claim 1 or claim 2, wherein the at least one photosensitive sensor pixel (121, 121') is further configured to increase a proportion of the charge carriers stored in the at least one charge storage (123, 125) with increasing distance of the optical sensor (120) from an object in the scene causing the reflected light (103). [4] The device according to one of claims 1 to 3, wherein the at least one photosensitive sensor pixel is further configured to selectively prevent the charge carriers from reaching the at least one charge storage (123, 125) based on at least one received drive signal. [5] The device according to any one of claims 1 to 4, further comprising: processing circuitry configured to determine a reflectivity value indicative of a reflectivity of an object in the scene based on an output value of the at least one photosensitive sensor pixel (121, 121') for the measurement. [6] The device according to one of claims 1 to 5, wherein the optical sensor (120) is a time-of-flight, ToF, sensor and wherein the measurement is a ToF measurement. [7] The device according to claim 6, wherein a correlation function of the at least one photosensitive sensor pixel (121, 121') for the ToF measurement increases strictly monotonically over the distance within the measuring range of the ToF sensor. [8] The device according to claim 7, wherein the correlation function indicates the distance-dependent correlation of the reflected light (103) with one or more control signals and without taking into account an intensity of the reflected light (103), wherein the at least one photosensitive sensor pixel (121, 121') is controlled based on the one or more control signals. [9] The device according to claim 7 or claim 8, wherein the correlation function increases quadratically with the distance within the measurement range. [10] The device according to one of claims 6 to 9, wherein the at least one photosensitive sensor pixel (121') is formed: selectively storing the charge carriers in at least two charge storage devices (123, 125) of the photosensitive sensor pixel (121') during the ToF measurement; and to selectively prevent the charge carriers from reaching the at least two charge storage devices (123, 125) during the ToF measurement. [11] The device according to claim 10, wherein an output value of the at least one photosensitive sensor pixel (121') for the ToF measurement is based on a charge difference between charges stored in the at least two charge storages (123, 125). [12] The device according to claim 5 and any one of claims 6 to 11, wherein the ToF sensor is further configured to perform one or more second ToF measurements before the ToF measurement is performed, and wherein the processing circuitry is configured: determine a distance value indicative of a distance to the object based on an output of the ToF sensor for the one or more second ToF measurements; and Based on the distance value, adjust the measuring range of the ToF sensor for the ToF measurement to include the object. [13] The device according to one of claims 6 to 12, wherein the at least one photosensitive sensor pixel (121') is configured to selectively prevent the charge carriers from reaching the at least one charge storage (123, 125) during the ToF measurement by selectively diverting the charge carriers. [14] The device according to one of claims 1 to 4, wherein the optical sensor (120) is an image sensor or a dynamic vision sensor. [15] A method (600) for capturing a scene, the method comprising: illuminating (602) the scene with modulated light using a lighting element; Receiving (604) light reflected from the scene at an optical sensor, the optical sensor comprising at least one photosensitive sensor pixel; Controlling (606, 608) the at least one photosensitive sensor pixel to: Storing charge carriers generated in the photosensitive sensor pixel by the reflected light in at least one charge storage device of the photosensitive sensor pixel during a measurement; and selectively preventing the charge carriers from reaching the at least one charge storage during the measurement in order to strictly monotonically increase a sensitivity of the at least one photosensitive sensor pixel over the distance for reflected light originating from within a measuring range of the optical sensor. [16] The method (600) of claim 15, wherein the sensitivity increases quadratically over the distance within the measurement range. [17] The method (600) of claim 15 or claim 16, further comprising: Determining a reflectivity value indicative of a reflectivity of an object in the scene based on an output value of the at least one photosensitive sensor pixel for the measurement. [18] The method (600) according to any one of claims 15 to 17, wherein the optical sensor is a time-of-flight (ToF) sensor and wherein the measurement is a ToF measurement. [19] The method (600) according to claim 17 or claim 18, further comprising: Performing one or more second ToF measurements with the ToF sensor before performing the ToF measurement; Determining a distance value indicative of a distance to the object based on an output of the ToF sensor for the one or more second ToF measurements; and Adjust, based on the distance value, the measurement range of the ToF sensor for the ToF measurement to include the object. [20] The method (600) of claim 18 or claim 19, wherein controlling (608) the at least one photosensitive sensor pixel to selectively prevent the charge carriers from reaching the at least one charge storage during the ToF measurement comprises controlling the at least one photosensitive sensor pixel to selectively drain the charge carriers.

Images