Time-of-flight camera with spot illumination

Abstract

A time-of-flight camera has an illumination device for emitting pulsed or / and modulated light, in which light is emitted in the form of a dot pattern or a multiplicity of light spots, a reception sensor having a plurality of pixels for receiving the emitted dot pattern or the emitted light spots, and a control device for controlling the illumination device, wherein the size of the pixels is smaller than the extension range of a single received light spot, and wherein the extension range of the light spot is specified in such a way that the received light spot illuminates a plurality of pixels,-an evaluation unit which is designed to evaluate only pixels in the region of the received light spot for distance measurements, the pixels are located in a predetermined value domain of a pixel measurement parameter, and the remaining pixels are rejected or endowed with negligible weights.

Description

[0001] The present invention relates to a time-of-flight camera of the type described in claim 1.

[0002] Time-of-flight cameras are intended to encompass all time-of-flight camera systems or 3D-TOF camera systems that obtain distance directly from light time-of-flight (dTOF) or from the phase shift of emitted and received radiation (iTOF). Particularly suitable as time-of-flight cameras or 3D-TOF cameras are PMD cameras with a light mixing detector (PMD), as described primarily in DE 19704 496 A1, or cameras with a SPAD array as the receiving sensor.

[0003] The purpose of this invention is to improve the measurement accuracy of time-of-flight cameras.

[0004] This objective is achieved by the time-of-flight camera of the present invention.

[0005] An advantageous time-of-flight camera has... - An illumination device for emitting pulsed and / or modulated light. The light is emitted in the form of dot patterns or numerous light spots. - A receiving sensor having multiple pixels for receiving the emitted dot pattern or the emitted light spot. The size of the pixel is smaller than the extent of a single received light spot. Furthermore, the extent of the light spot is defined such that the received light spot illuminates multiple pixels. - An evaluation unit, designed to evaluate only pixels within the region of the received light spot for distance measurements, provided that these pixels fall within a predetermined range of pixel measurement parameters. Remove the remaining pixels or assign them negligible weights.

[0006] The term "pixel measurement parameter" is intended to encompass all measurement parameters characterizing the amount of light accumulated at a pixel, such as the charge at the integration node, the applied voltage, or other parameters derived therefrom.

[0007] The operation involves time-of-flight cameras that determine distances directly or indirectly from the time of flight of light, and in this case, preferably emit pulsed and / or modulated light.

[0008] The advantage of the time-of-flight camera of the present invention is that, by specifying appropriate limits, only pixels that can make a significant contribution to distance measurement are used for distance measurement. In particular, if a pixel at the center of the spot is saturated or exceeds the saturation threshold, it can be excluded during processing; however, the unsaturated edges of the spot can still be used for evaluation.

[0009] Preferably, the processing unit is designed to apply a correction to the distance value in order to determine the distance value, the correction taking into account the non-uniform distribution of a specific spot measurement parameter within the corresponding spot and the removal of saturated pixels.

[0010] As parameters for measuring light spots, for example (but not limited to) phase position, time of flight of light, amplitude, etc., can be considered.

[0011] The advantage of this procedure is that it allows for the spatial distribution of light / spot measurement parameters across the spot's extension range to be considered in distance determination. For example, in some focused projectors, due to hardware limitations, the phase position at the spot center differs from that at the spot edge. This results in non-uniformity in the phase observed within a single spot, even when measuring a flat wall surface. Therefore, discarding pixels at the spot center without calibration can introduce systematic errors.

[0012] Advantageously, the correction is a weighted average of the non-uniformity and the location and / or number of saturated pixels that have been removed.

[0013] Based on the knowledge of the location and / or number of pixels being removed, it is feasible to better define the weight distribution.

[0014] These weights can be stored, for example, as lookup tables and / or stored in memory as functions.

[0015] The attached diagram shows Figure 1 Shooting scenes using traditional lighting, Figure 2 Shooting scenes using spotlight lighting. Figure 3 The pixel region near the received light spot with unsaturated pixels. Figure 4 The pixel region near the received light spot with saturated pixels. Figure 5 Distance error related to the number of saturated pixels.

[0016] In active lighting shooting methods, achieving a very high dynamic range for a scene is often challenging. This is especially true for Time-of-Flight (ToF) systems, where it is crucial that pixel measurements are not saturated, otherwise distance errors will occur.

[0017] The term "saturated pixel" does not necessarily refer to the pixel with the maximum possible measurement value; it can also refer to a pixel whose measurement is located in the nonlinear region of the sensor. When multiple measurements are combined, the pixel that has been saturated at least once is also referred to as a saturated pixel.

[0018] This problem can be cleverly solved by using spot lighting. In this case, only individual points in the scene are illuminated, and distance is evaluated only at these points.

[0019] Figure 1 The scene is shown exemplarily under uniform lighting. Figure 2 The scene is shown with spot lighting.

[0020] In a spotting system, a spot typically covers multiple pixels. The spot is characterized on the sensor by its center (e.g., the midpoint, centroid, or brightest pixel) and the area surrounding the center (e.g., a circular area with a radius of 3.5 pixels or a 9×9 pixel square area). For evaluation, for example, a weighted average of the required data (e.g., phase, amplitude, distance, raw data) output for each area surrounding the spot is first performed, followed by subsequent processing in the manner typical for this method. The exposure time is set in this case such that there is no pixel saturation or as little pixel saturation as possible.

[0021] In the context of this invention, an even longer exposure time can also be selected. The exposure time can be set, for example, such that at least a portion of the area surrounding the light spot is unsaturated, at least for scene-related light spots.

[0022] Figure 3 An example is shown of a possible region surrounding a light spot centered at (4,4). In this example, the region surrounding the light spot is completely unsaturated.

[0023] Figure 4 This shows the possible area around the light spot, where five pixels are saturated in the area around the light spot. mark.

[0024] In this case, saturated pixels are not considered when performing weighted averaging, or their weights are set to negligible small values. Since high signal values ​​typically appear in the area around partially saturated spots, and the remaining pixels have sufficient signal, the missing pixels have almost no impact.

[0025] However, in general, such as in time-of-flight cameras or spot projection systems, the phase distribution within the spot is not uniform due to hardware limitations. Therefore, pixel removal affects the accuracy of distance measurements. For example, if the phase at the spot center is less than the phase at the spot edge, removing pixels closer to the center results in an overestimation of the phase weighted average. To compensate for this error, two possibilities are proposed below.

[0026] 1. Number of saturated pixels In this case, a reference measurement is performed that covers as many saturation levels (the number of saturated pixels in the area surrounding the spot) as possible in subsequent measurements. Now, after applying standard calibration, the distance error is determined and plotted against the saturation levels. Based on this, a general relationship describing this correlation is derived (e.g., through polynomial fitting). See [link to relevant documentation]. Figure 5 During operation, this relationship is used to correct distance errors for each light spot. This relationship can be determined individually for each light spot, be the same for a group of light spots, or be uniformly applied to all light spots.

[0027] 2. Individual pixel correction Based on theoretical analysis or reference measurements, the functional relationship between the phase difference and the position relative to the spot center is determined. Preferably, the spot center is determined with sub-pixel accuracy in this case. More preferably, lens distortion is also considered in this case. The phase correction for the corresponding spot is determined by weighted averaging of the phase differences of the rejected pixels or the pixels actually used. The method is then continued using the corrected phase.

[0028] These possibilities are merely examples. The invention is not limited thereto.

[0029] Distance and phase are used only for illustrative purposes. It is also conceivable to correct for, or based on, the amplitude, unmodulated intensity, the raw data, or the measured (amplitude normalized) real / imaginary parts (I / Q components).

Claims

1. A time-of-flight camera having - an illumination device for emitting pulsed or / and modulated light, wherein the light is emitted in the form of a dot pattern or a multitude of light spots, - a receiving sensor having a plurality of pixels for receiving the emitted dot pattern or the emitted light spots, wherein the size of the pixels is smaller than the extension of a single received light spot, and wherein the extension of the light spots is so defined that a received light spot illuminates a plurality of pixels, - an evaluation unit designed to evaluate, for a distance measurement, only pixels within the area of a received light spot that lie within a predetermined value range of a pixel measurement parameter, the remaining pixels being either rejected or given a negligible weight.

2. The time-of-flight camera according to claim 1, wherein the processing unit is designed to apply, for determining a distance value, a correction of the distance value that takes into account the non-uniform distribution of a certain light spot measurement parameter within the respective light spot and the rejection of the rejected pixels.

3. The time-of-flight camera of claim 2, wherein, the correction results from a weighted average of the non-uniformity and the position of the rejected pixels.

4. The time-of-flight camera of claim 2 or 3, wherein, the correction results from the number of the rejected pixels.

5. The time-of-flight camera according to any of claims 2 to 4, characterized in that, the correction or at least one correction parameter depends on the respective light spot or the position of the respective light spot on the sensor.

6. The time-of-flight camera according to any of the preceding claims, wherein, the light is emitted in a modulated manner, and the time-of-flight camera is designed to determine a distance value from the phase shift between the emitted and the received light.

7. The time-of-flight camera according to any one of claims 1 to 5, wherein, the light is emitted in a pulsed manner, and the receiving sensor has a plurality of pixels that are designed as SPAD pixels.

Images