Time-of-flight camera with spot illumination

The time-of-flight camera system addresses measurement accuracy issues by using spot lighting and selective pixel evaluation with corrections for inhomogeneous light distribution, improving distance determination in high dynamic range scenes.

WO2024245650A1PCT designated stage expired Publication Date: 2024-12-05IFM ELECTRONIC GMBH +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2024/061365
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-04-16
Filing Date
2024-04-25
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Time-of-flight cameras face challenges in measurement accuracy due to high dynamic range scenes causing pixel saturation, which leads to distance errors, especially when using conventional lighting that results in inhomogeneous light distribution within the spot area.

Method used

A time-of-flight camera system employing spot lighting with a pulsed and/or modulated light emitting a dot pattern or spots, utilizing a reception sensor with smaller pixels and an evaluation unit that selectively evaluates only unsaturated pixels within a predetermined range, applying corrections for inhomogeneous spot measurement variables and omitting saturated pixels through weighted averaging or pixel-specific corrections.

Benefits of technology

This approach enhances measurement accuracy by focusing on relevant pixels, reducing systematic errors from saturated pixels and inhomogeneous light distribution, allowing for precise distance determination even in high dynamic range scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024061365_05122024_PF_FP_ABST
    Figure EP2024061365_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The invention relates to a time-of-flight camera comprising an illumination means for emitting a pulsed and / or modulated light, wherein the light is emitted in the form of a dot pattern or a plurality of spots, comprising a receiving sensor with multiple pixels for receiving the emitted dot pattern or the emitted spots, wherein the size of the pixels is smaller than the extension of an individual received spot, and wherein the extension of the spot is determined such that the received spot illuminates multiple pixels, and comprising an evaluation unit which is designed in such a way that, for a distance measurement, only those pixels in the area of the received spot are evaluated which lie within a predefined value range of a pixel measurement variable, and the remaining pixels are omitted or given a negligible weighting.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Time-of-flight camera with spot lighting

[0002] The invention relates to a time-of-flight camera according to the preamble of claim 1.

[0003] The term "time-of-flight camera" encompasses all time-of-flight or 3D-TOF camera systems that determine distances directly from the time-of-flight (dTOF) or from the phase shift of emitted and received radiation (iTOF). PMD cameras with photonic mixer detectors (PMD), as described in DE 197 04 496 A1, among others, or cameras that use a SPAD array as the receiving sensor, are particularly suitable as time-of-flight or 3D-TOF cameras.

[0004] The object of the invention is to improve the measurement accuracy of a time-of-flight camera.

[0005] The problem is solved by the time-of-flight camera according to the invention.

[0006] A time-of-flight camera is advantageous

[0007] - with a lighting device for emitting pulsed and / or modulated light, wherein the light is emitted in the form of a dot pattern or a plurality of spots,

[0008] - with a receiving sensor with several pixels for receiving the emitted dot pattern or the emitted spots, wherein the size of the pixels is smaller than the extent of a single received spot, and wherein the extent of the spot is set such that the received spot illuminates several pixels,

[0009] - with an evaluation unit which is designed in such a way that for a distance measurement only the pixels in the area of ​​the received spot which lie within a predetermined value range of a pixel measurement quantity are evaluated, the remaining pixels are omitted or given a negligible weight.

[0010] The term pixel measurement is intended to include all measurements that characterize the amount of light accumulated at the pixel, such as the charge at the integration node, the applied voltage or other quantities derived from them.

[0011] The procedure concerns time-of-flight cameras that determine distances directly or indirectly from a time-of-flight and preferably emit pulsed and / or modulated light.

[0012] The time-of-flight camera according to the invention has the advantage that, by setting a suitable threshold, only those pixels that can make a relevant contribution to the distance measurement are used for distance measurement. In particular, if the pixels in the center of a spot are saturated or above a saturation threshold, these can be excluded from the evaluation, while the unsaturated edge of the spot can still be used for evaluation.

[0013] Preferably, the evaluation unit is designed such that a correction of the distance values ​​is applied to determine distance values, which correction takes into account an inhomogeneous distribution of certain spot measurement variables within a respective spot and the omission of the saturated pixels.

[0014] Spot measurement variables that can be taken into account include, for example, phase position, light propagation time and amplitude, but are not limited to this list.

[0015] This approach has the advantage that the spatial distribution of light or spot measurement variables within the spot's extent can be taken into account for distance determination. For example, with certain spot projectors, the phase position in the spot center is different than at the spot's edge due to hardware limitations. This leads to inhomogeneity in the observed phase within a spot, even when measuring a flat wall. Omitting the pixels in the spot center would therefore cause a systematic error without correction.

[0016] It is advantageous if the correction results from a weighted average of the inhomogeneity and the position and / or number of omitted, saturated pixels.

[0017] Knowing the position and / or number of omitted pixels makes it possible to better adjust the distribution of weights.

[0018] The weights can be stored, for example, in a look-up table and / or as a function in a memory.

[0019] Figure 1 shows a shot of a scene with conventional lighting,

[0020] Figure 2 shows a shot of a scene with spot lighting,

[0021] Figure 3 Pixel area in the vicinity of a received spot with non-saturated pixels,

[0022] Figure 4 Pixel area in the vicinity of a received spot with saturated pixels,

[0023] Figure 5 Distance error in relation to the number of saturated pixels.

[0024] In active illumination imaging techniques, the very high dynamic range of the scenes often presents a challenge. Especially for ToF systems, it is essential that the pixel measurements are not saturated, as this causes distance errors.

[0025] Saturated pixels don't necessarily refer to pixels with the maximum possible measured value; they can also refer to pixels where the measurement is in the nonlinear range of the sensor. If multiple measurements are combined, pixels where at least one measurement is saturated can also be described as saturated.

[0026] This problem can be elegantly solved by using spot lighting. This method illuminates the scene only at specific points, and distances are only evaluated at these points.

[0027] Figure 1 shows an example of a scene that is homogeneously illuminated and Figure 2 shows a scene with spot lighting.

[0028] In spot systems, a spot usually covers several pixels. A spot is characterized on the sensor by a spot center (e.g., the center, center of gravity, or the brightest pixel of the spot) and a spot surrounding this center (e.g., a circular area with a radius of 3.5 pixels or a square with 9x9 pixels). For evaluation, the data required for output (e.g., phases, amplitudes, distances, raw data) are weighted averaged for each spot surroundings before being further processed in the usual way for the process. The exposure time is set so that no or as few pixels as possible are saturated.

[0029] Within the scope of the invention, longer exposure times can also be selected. These can, for example, be set so that at least part of the spot surroundings is desaturated, at least for the spots relevant to the scene.

[0030] Figure 3 shows an example of a possible spot environment with a spot center at (4, 4). In this example, the spot environment is completely unsaturated.

[0031] Figure 4 shows a possible spot environment where five pixels within the spot environment are saturated, here marked with 0.

[0032] In weighted averaging, saturated pixels are ignored, or their weights are set to a negligible value. Since partially saturated spot environments typically have high signal values, the remaining pixels have sufficient signal, and the missing pixels are of little importance.

[0033] In general, however, the phase distribution within a spot is not homogeneous due to hardware limitations, for example, in time-of-flight cameras or spot projectors. Therefore, omitting pixels has implications for the accuracy of the distance measurement. For example, if the phase is smaller at the center of the spot than at the edge of the spot, omitting pixels close to the center results in a larger weighted average phase value. To compensate for this error, two options are proposed below.

[0034] 1 . Number of saturated pixels

[0035] A reference measurement is taken that covers, as far as possible, all saturation levels (number of saturated pixels in the spot surroundings) that may occur in subsequent measurements. The distance errors are then determined after applying the usual calibration and plotted against the saturation level. Based on this, a general relationship is derived that describes this relationship (e.g., by polynomial fitting). See Figure 5

[0036] Based on this relationship, the distance error is corrected for each spot during operation. This relationship can be determined individually for each spot, be identical for groups of spots, or apply equally to all spots. 2. Pixel-specific correction

[0037] Based on theoretical considerations or reference measurements, the phase difference is determined as a function of the position relative to the spot center. Preferably, the spot center is determined with sub-pixel accuracy. Lens distortions are also preferably taken into account. A weighted average of these phase differences for the omitted or actually used pixels is used to determine a phase correction for the respective spot. The process continues with the corrected phase.

[0038] These possibilities serve merely as examples. The invention is not limited to them.

[0039] Distance and phase are also used only to illustrate the examples. Corrections based on, for example, amplitude, unmodulated intensity, raw data, or (amplitude-normalized) real / imaginary parts (I / Q components) of the measurement are also conceivable.

Claims

Patent claims 1 . Time-of-flight camera - with a lighting device for emitting pulsed and / or modulated light, wherein the light is emitted in the form of a dot pattern or a plurality of spots, - with a receiving sensor with several pixels for receiving the emitted dot pattern or the emitted spots, wherein the size of the pixels is smaller than the extent of a single received spot, and wherein the extent of the spot is set so that the received spot illuminates several pixels, - with an evaluation unit which is designed in such a way that for a distance measurement only the pixels in the area of ​​the received spot which lie within a predetermined value range of a pixel measurement quantity are evaluated, the remaining pixels are omitted or given a negligible weight.

2. Time-of-flight camera according to claim 1, wherein the evaluation unit is designed such that a correction of the distance values ​​is applied to determine distance values, which correction takes into account an inhomogeneous distribution of certain spot measurement variables within a respective spot and the omission of the omitted pixels.

3. Time-of-flight camera according to claim 2, wherein the correction results from a weighted average of the inhomogeneity and the position of the omitted pixels.

4. A time-of-flight camera according to claim 2 or 3, wherein the correction results from the number of omitted pixels.

5. Time-of-flight camera according to one of claims 2 to 4, wherein the correction or at least one correction parameter depends on the respective spot or on the position of the respective spot on the sensor.

6. Time-of-flight camera according to one of the preceding claims, in which the light is emitted in a modulated manner, and the time-of-flight camera is designed such that distance values ​​are determined from a phase shift between the emitted and received light.

7. Time-of-flight camera according to one of claims 1 to 5, wherein the light is emitted in pulses and the receiving sensor has a plurality of pixels which are designed as SPAD pixels.

Citation Information

Patent Citations

  • Method and device for determining the phase and / or amplitude information of an electromagnetic wave

    DE19704496A1

  • Optoelectronic sensor for detecting an object

    DE202016104285U1

  • Focus scanning apparatus recording color

    EP4094719A1

  • Light detector and distance measuring device

    US20220397650A1

  • Glare mitigation in lidar applications

    WO2020047248A1