Time-of-flight camera with spot lighting
The time-of-flight camera system addresses pixel saturation and inhomogeneous light distribution by using spot lighting and pixel corrections, enhancing measurement accuracy and reducing systematic errors.
Patent Information
- Application Number
- DE102024110657
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-26
- Filing Date
- 2024-04-16
- Publication Date
- 2024-11-28
AI Technical Summary
Existing time-of-flight cameras face challenges in achieving accurate distance measurements due to pixel saturation and inhomogeneous light distribution within the spot, leading to systematic errors.
A time-of-flight camera system that uses pulsed and/or modulated light in a dot pattern or spots, evaluates only relevant pixels within a predetermined value range, applies corrections for saturated pixels, and employs weighted averaging or pixel-specific corrections to account for inhomogeneous light distribution.
Enhances measurement accuracy by excluding saturated pixels and correcting for inhomogeneous light distribution, thereby reducing systematic errors and improving distance determination.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a time-of-flight camera according to the preamble of claim 1.
[0002] The term "time-of-flight camera" encompasses all time-of-flight or 3D-TOF camera systems that determine distances directly from the time-of-flight (dTOF) or from the phase shift of emitted and received radiation (iTOF). PMD cameras with photonic mixer detectors (PMD), as described in DE 197 04 496 A1, among others, or cameras that use a SPAD array as the receiving sensor, are particularly suitable as time-of-flight or 3D-TOF cameras.
[0003] The object of the invention is to improve the measurement accuracy of a time-of-flight camera.
[0004] The problem is solved by the time-of-flight camera according to the invention.
[0005] A time-of-flight camera is advantageous - with an illumination for emitting pulsed and / or modulated light, where the light is emitted in the form of a dot pattern or a multitude of spots, - with a receiving sensor with several pixels for receiving the emitted dot pattern or spots, where the size of the pixels is smaller than the extent of a single received spot, and wherein the extent of the spot is determined such that the received spot illuminates several pixels, - with an evaluation unit which is designed in such a way that for a distance measurement only the pixels in the area of the received spot which lie within a predetermined value range of a pixel measurement quantity are evaluated, the remaining pixels are omitted or given a negligible weight.
[0006] The term pixel measurement is intended to include all measurements that characterize the amount of light accumulated at the pixel, such as the charge at the integration node, the applied voltage or other quantities derived from them.
[0007] The procedure concerns time-of-flight cameras that determine distances directly or indirectly from a time-of-flight and preferably emit pulsed and / or modulated light.
[0008] The time-of-flight camera according to the invention has the advantage that, by setting a suitable threshold, only those pixels that can make a relevant contribution to the distance measurement are used for distance measurement. In particular, if the pixels in the center of a spot are saturated or above a saturation threshold, these can be excluded from the evaluation, while the unsaturated edge of the spot can still be used for evaluation.
[0009] Preferably, the evaluation unit is designed such that a correction of the distance values is applied to determine distance values, which correction takes into account an inhomogeneous distribution of certain spot measurement variables within a respective spot and the omission of the saturated pixels.
[0010] Spot measurement variables that can be taken into account include, for example, phase position, light propagation time and amplitude, but are not limited to this list.
[0011] This approach has the advantage that the spatial distribution of light or spot measurement variables within the spot's extent can be taken into account for distance determination. For example, with certain spot projectors, the phase position in the spot center is different than at the spot's edge due to hardware limitations. This leads to inhomogeneity of the observed phase within a spot, even when measuring a flat wall. Omitting the pixels in the spot center would therefore cause a systematic error without correction.
[0012] It is advantageous if the correction results from a weighted average of the inhomogeneity and the position and / or number of omitted, saturated pixels.
[0013] Knowing the position and / or number of omitted pixels makes it possible to better adjust the distribution of weights.
[0014] The weightings can, for example, be stored in a look-up table and / or as a function in a memory.
[0015] It shows Fig. 1 a photograph of a scene with conventional lighting, Fig. 2 a shot of a scene with spot lighting, Fig. 3 Pixel area in the vicinity of a received spot with non-saturated pixels, Fig. 4 Pixel area in the vicinity of a received spot with saturated pixels, Fig. 5 Distance error in relation to the number of saturated pixels.
[0016] In active illumination imaging techniques, the very high dynamic range of the scenes often presents a challenge. Especially for ToF systems, it is essential that the pixel measurements are not saturated, as this causes distance errors.
[0017] Saturated pixels don't necessarily refer to pixels with the maximum possible measured value; they can also refer to pixels where the measurement is in the nonlinear range of the sensor. If multiple measurements are combined, pixels where at least one measurement is saturated can also be described as saturated.
[0018] This problem can be elegantly solved by using spot lighting. This method illuminates the scene only at specific points, and distances are only evaluated at these points.
[0019] Fig. 1 shows an example of a scene that is homogeneously illuminated and Fig. 2 a scene with spot lighting.
[0020] In spot systems, a spot usually covers several pixels. A spot is characterized on the sensor by a spot center (e.g., the center, center of gravity, or the brightest pixel of the spot) and a spot surrounding this center (e.g., a circular area with a radius of 3.5 pixels or a square with 9×9 pixels). For evaluation, the data required for output (e.g., phases, amplitudes, distances, raw data) are weighted averaged for each spot surroundings before being further processed in the usual way for the process. The exposure time is set so that no or as few pixels as possible are saturated.
[0021] Within the scope of the invention, longer exposure times can also be selected. These can, for example, be set so that at least part of the spot surroundings is desaturated, at least for the spots relevant to the scene.
[0022] Fig. Figure 3 shows an example of a possible spot environment with a spot center at (4, 4). In this example, the spot environment is completely unsaturated.
[0023] Fig. Figure 4 shows a possible spot environment where five pixels within the spot environment are saturated, here marked with ⊗.
[0024] In weighted averaging, saturated pixels are ignored, or their weights are set to a negligible value. Since partially saturated spot environments typically have high signal values, the remaining pixels have sufficient signal, and the missing pixels are of little importance.
[0025] In general, however, the phase distribution within a spot is not homogeneous due to hardware limitations, for example, in time-of-flight cameras or spot projectors. Therefore, omitting pixels has implications for the accuracy of the distance measurement. For example, if the phase is smaller at the center of the spot than at the edge of the spot, omitting pixels close to the center results in a larger weighted average phase value. To compensate for this error, two options are proposed below. 1. Number of saturated pixels
[0026] A reference measurement is taken that covers, if possible, all saturation levels (number of saturated pixels in the spot surroundings) that may occur in subsequent measurements. The distance errors are then determined after applying the usual calibration and plotted against the saturation level. Based on this, a general relationship is derived that describes this relationship (e.g., by polynomial fitting). See Fig. 5
[0027] Based on this relationship, the distance error is corrected for each spot during operation. This relationship can be determined individually for each spot, be identical for groups of spots, or apply equally to all spots. 2. Pixel-specific correction
[0028] Based on theoretical considerations or reference measurements, the phase difference is determined as a function of the position relative to the spot center. Preferably, the spot center is determined with sub-pixel accuracy. Lens distortions are also preferably taken into account. A weighted average of these phase differences for the omitted or actually used pixels is used to determine a phase correction for the respective spot. The process continues with the corrected phase.
[0029] These possibilities serve merely as examples. The invention is not limited to them.
[0030] Distance and phase are also used only to illustrate the examples. Corrections based on, for example, amplitude, unmodulated intensity, raw data, or (amplitude-normalized) real / imaginary parts (I / Q components) of the measurement are also conceivable. QUOTES CONTAINED IN THE DESCRIPTION
[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature
[0000] DE 197 04 496 A1
[0002]
Claims
[1] Time-of-flight camera - with an illumination for emitting pulsed and / or modulated light, where the light is emitted in the form of a dot pattern or a multitude of spots, - with a receiving sensor with several pixels for receiving the emitted dot pattern or spots, where the size of the pixels is smaller than the extent of a single received spot, and wherein the extent of the spot is determined such that the received spot illuminates several pixels, - with an evaluation unit which is designed in such a way that for a distance measurement only those pixels in the area of the received spot are evaluated which lie within a predetermined value range of a pixel measurement variable, the remaining pixels are omitted or given a negligible weight. [2] Time-of-flight camera according to claim 1, wherein the evaluation unit is designed such that a correction of the distance values is applied to determine distance values, which correction takes into account an inhomogeneous distribution of certain spot measurement variables within a respective spot and the omission of the omitted pixels. [3] Time-of-flight camera according to claim 2, wherein the correction results from a weighted average of the inhomogeneity and the position of the omitted pixels. [4] Time-of-flight camera according to claim 2 or 3, wherein the correction results from the number of omitted pixels. [5] Time-of-flight camera according to one of claims 2 to 4, in which the correction or at least one correction parameter depends on the respective spot or on the position of the respective spot on the sensor. [6] Time-of-flight camera according to one of the preceding claims, in which the light is emitted in a modulated manner, and the time-of-flight camera is designed such that distance values are determined from a phase shift between the emitted and received light. [7] Time-of-flight camera according to one of claims 1 to 5, in which the light is emitted in pulses and the receiving sensor has a plurality of pixels which are designed as SPAD pixels.
Citation Information
Patent Citations
Method and device for determining the phase and / or amplitude information of an electromagnetic wave
DE19704496A1
