Light time-of-flight camera system

Abstract

Time-of-flight camera system (1) with an illumination (10) for emitting a modulated light (Sp1) with a predetermined wavelength and a time-of-flight camera (20) with a time-of-flight sensor (22) having several time-of-flight pixels (23), which are configured as PMD pixels with modulation gates (Gam, Gbm) and at least two integration nodes (Ga, Gb), with an evaluation unit designed in such a way that a distance value (d) is determined on the basis of a phase shift (Δφ) between the emitted (Sp1) and a received light (Sp2), where, to determine the phase shift (Δφ), a difference in the charges applied to the integration nodes (Ga, Gb) after an integration time or their voltage equivalent is formed, characterized by that the camera (20) has an edge or bandpass filter which is transparent to the specified wavelength of the emitted light, and that the illumination (10) is designed such that, in addition to emitting modulated light with a basic amplitude (A BB ) a modulated light in the form of a dot pattern (50) is emitted, the dots (51) of which have an amplitude (A B ) exhibit a value greater than the baseline amplitude (A BB ), where light emission between the points of the dot pattern (50) is not reduced to zero, but only to the basic amplitude (A) BB ) falls off, where the number of points (51) in the dot pattern (50) is less than the number of light time-of-flight pixels (23) of the light time-of-flight sensor (22), and the evaluation unit is designed such that only the pixels (23) of the time-of-flight sensor (22) whose detected useful light amplitude (A) are used to determine a distance value S ) a given amplitude limit (A SG exceed, where the amplitude limit (A SG ) or / and the basic amplitude (A BB ) are defined such that in a near range the basic amplitude (A BB ) the amplitude limit (A SG ) exceeds and falls below in a distant range.

Description

[0001] The invention relates to a time-of-flight camera system according to the preamble of claim 1.

[0002] Time-of-flight camera systems are intended to encompass, in particular, all time-of-flight or 3D-TOF camera systems that derive time-of-flight information from the phase shift of emitted and received radiation. PMD cameras with photomixing detectors (PMDs), such as those described in DE 197 04 496 A1 and available, for example, from companies like 'ifm electronic GmbH' or 'pmdtechnologie ag' as Frame-Grabber O3D or CamCube, are particularly suitable as time-of-flight or 3D-TOF cameras. The PMD camera allows, in particular, a flexible arrangement of the light source and the detector, which can be housed in a single unit or mounted separately.

[0003] Furthermore, DE 10 2011 089 629 A1 discloses a time-of-flight camera in which time-of-flight pixels are grouped together depending on the grayscale values ​​captured. A common distance value is determined for the grouped pixels. Preferably, only time-of-flight pixels whose amplitudes exceed a defined threshold are used.

[0004] EP 2 947 482 A2 describes an image sensor device for optically monitoring a field of view using spot-matrix illumination. Instead of illuminating the field of view uniformly, an array of focused light points is projected across it, thus reducing the required illumination energy. The light points can be distributed across the field of view at varying intensities to reliably detect both dark and bright objects. The intensities of the light points can be modulated based on the conditions of the field of view. This enables improved detection and localization of objects with varying reflectivity and distances within a single image frame.

[0005] From US patent 2006 / 0227317A1, a lidar system is known with an illumination system that has a beam shaping element in the beam path, with which objects or object points in the camera's field of view can be selectively illuminated.

[0006] German patent DE 10 2015 205 927 A1 discloses a distance measurement system comprising a time-of-flight sensor with a pixel array and an illumination system for emitting modulated light. The illumination system is designed to emit at least one geometric structure with modulated light, which is detected upon reflection from an object. An evaluation unit determines the distance value by triangulating the detected structure and analyzing the phase shift of the light. The system can emit multiple optical structures with different phase angles and form the structure using light streaks or point clouds. The evaluation unit outputs distance values ​​only if the difference is acceptable.

[0007] US Patent 8,982,363 B2 describes a method and apparatus for capturing depth information from a three-dimensional scene. The method includes illuminating the scene, receiving reflected light through at least one detector, and processing the resulting signals. A pseudorandomly generated series of spatial light modulation patterns is used to modulate the light pulses either before or after reflection. The digital samples are processed to estimate depth impulse responses and create a depth map of the scene. The technique enables improved spatial resolution and can be used in various applications, such as mobile devices.

[0008] German patent application DE 10 2010 039 092 A1 discloses a method for determining the distance of an object from a vehicle using an intensity pattern recorded in the infrared wavelength range. The method comprises receiving an intensity pattern representing the vehicle's surroundings, determining the spatial extent of the intensity pattern area on the object's surface, and calculating the distance based on this extent. A control unit performs the steps of the method. The method enables model-free measurement of objects and living beings in road traffic and utilizes existing infrared lighting in the vehicle to ensure precise distance measurement.

[0009] The object of the invention is to improve the distance measurement of a time-of-flight camera system.

[0010] The problem is advantageously solved by the time-of-flight camera system according to claim 1.

[0011] Advantageously, a time-of-flight camera system is provided, comprising an illumination for emitting modulated light with a predetermined wavelength and a time-of-flight camera with a time-of-flight sensor having several time-of-flight pixels designed as PMD pixels with modulation gates and at least two integration nodes, with an evaluation unit designed such that a distance value is determined based on a phase shift between the emitted and the received modulated light, wherein, to determine the phase shift, a difference in the charges applied to the integration nodes after an integration time or their voltage equivalent is calculated. wherein the camera has a spectral edge or bandpass filter that is transparent to the specified wavelength of the emitted light, and the illumination is designed such that it does not emit light uniformly and homogeneously, but forms a dot pattern whose points have a (modulation) amplitude that is greater than a base amplitude, wherein the light between the points of the dot pattern does not fall below the base amplitude, and wherein the number of points in the dot pattern is less than the number of time-of-flight pixels of the time-of-flight sensor, and the evaluation unit is designed in such a way that only the pixels of the time-of-flight sensor are used to determine a distance value, whose detected useful light amplitude (after demodulation in the pmd receiver) exceeds a predetermined amplitude limit.

[0012] This approach has the advantage that the basic amplitude of the illumination for a near range and / or highly reflective objects has a sufficiently high signal strength, so that preferably a sufficient brightness or useful light amplitude can be measured at all pixels on the time-of-flight sensor, which exceeds the specified amplitude limit at all pixels.

[0013] For distant objects and / or poorly reflective objects for which the base amplitude no longer provides sufficient brightness, the light points of the dot pattern emitted with a larger amplitude can be used for distance determination, accepting a lower lateral resolution.

[0014] It is particularly advantageous if the evaluation unit is designed in such a way that the time-of-flight sensor is divided into several sub-pixel areas when the amplitude limit is no longer reached by a minimum number of pixels, whereby only the time-of-flight pixel with maximum brightness in this sub-pixel area is used to determine a distance value.

[0015] This approach has the advantage that for the respective sub-pixel area, only the best illuminated pixel is evaluated in order to obtain the best possible signal / noise ratio.

[0016] It is particularly advantageous if the ratio of the number of emitted light points to the number of time-of-flight pixels of the time-of-flight sensor is 1:4 or less.

[0017] The invention will now be explained in more detail using exemplary embodiments and with reference to the drawings.

[0018] They show: Fig. 1 schematically a time-of-flight camera system, Fig. 2 a modulated integration of generated charge carriers, Fig. 3 a time-of-flight camera system according to the invention, Fig. 4 different dot pattern variations, Fig. 5 a cross-section of a light distribution through a dot pattern, Fig. 6 a cross-section of a light distribution through a dot pattern with a base amplitude Fig. 7 a cross-section of a captured useful light amplitude with sufficient basic amplitude, Fig. 8 a cross-section of a detected useful light amplitude with an insufficient detected basic amplitude.

[0019] In the following description of preferred embodiments, identical reference numerals denote identical or comparable components.

[0020] Fig. Figure 1 shows a measurement situation for an optical distance measurement with a time-of-flight camera, as is known, for example, from DE 197 04 496 A1.

[0021] The time-of-flight camera system 1 comprises a transmitter unit or illumination module 10 with an illumination 12 and an associated beam shaping optics 15, as well as a receiver unit or time-of-flight camera 20 with a receiving optics 25 and a time-of-flight sensor 22.

[0022] The time-of-flight sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is in particular designed as a PMD sensor. The receiving optics 25 typically consist of several optical elements to improve the imaging properties. The beam-shaping optics 15 of the transmitting unit 10 can, for example, be designed as a reflector or lens optics. In a very simple embodiment, optical elements can optionally be omitted on both the receiving and transmitting sides.

[0023] The measuring principle of this arrangement is essentially based on the fact that, starting from the phase shift of the emitted and received light, the travel time and thus the distance traveled by the received light can be determined. For this purpose, the light source 12 and the light travel time sensor 22 are jointly supplied with a specific modulation signal M0 with a base phase φ0 via a modulator 30. In the illustrated example, a phase shifter 35 is also provided between the modulator 30 and the light source 12, with which the base phase φ0 of the modulation signal M0 of the light source 12 is shifted by defined phase positions φ var can be shifted. For typical phase measurements, phase positions of φ are preferably used. var = 0°, 90°, 180°, 270° used.

[0024] Depending on the set modulation signal, the light source 12 sends an intensity-modulated signal S p1with the first phase position p1 or p1 = φ0 + φ var off. This signal S p1 or, in the case shown, the electromagnetic radiation is reflected by an object 40 and arrives with a corresponding phase shift Δφ(t) due to the distance traveled. L ) with a second phase position p2 = φ0 + φ var + Δφ(t L ) as a received signal S p2 on the light time-of-flight sensor 22. In the light time-of-flight sensor 22, the modulation signal M is o with the received signal S p2 mixed, whereby the phase shift or the object distance d is determined from the resulting signal.

[0025] Infrared LEDs are preferably suitable as the illumination source or light source 12. Of course, other radiation sources in other frequency ranges are also conceivable, in particular light sources in the visible frequency range.

[0026] The basic principle of phase measurement is schematically represented in Fig. Figure 2 shows the time course of the modulation signal M0, which controls the lighting 12 and the light-time-of-flight sensor 22. The light reflected from object 40 arrives as the received signal S. p2 according to its light travel time t L phase-shifted Δφ(t L ) to the time-of-flight sensor 22. The time-of-flight sensor 22 collects the photonically generated charges q over several modulation periods in the phase of the modulation signal M0 in a first integration node, in particular accumulation gate Ga, and in a phase shifted by 180° M0 + 180° in a second integration node, in particular accumulation gate Gb. From the ratio of the charges qa, qb collected in the first and second gates Ga, Gb, the phase shift Δφ(t) can be determined. L ) and thus determine a distance d of the object.

[0027] Fig. Figure 3 shows an embodiment of the invention in which a dot pattern 50 is emitted instead of homogeneous light. The dot pattern 50 can be generated, for example, using diffractive optics 18. However, it is also conceivable to generate a suitable dot pattern 50 using several light sources, in particular VCSELs or LEDs. The objects 40 illuminated by the dot pattern 50 are detected by the camera 20.

[0028] The optics 25 of the camera 20 image the points 51 of the dot pattern 50 onto the time-of-flight sensor. The size of the points 51 in the emitted dot pattern 50 is adjusted such that, upon reception, a single point 51 of the dot pattern 50 preferably illuminates one pixel 23 of the time-of-flight sensor 22 completely.

[0029] Fig. Figure 4 shows possible interpretations of the dot pattern 50. Fig. Figure 4a shows a dot pattern 50 in which, for each 4x4 sub-pixel matrix 24, one pixel 23 is illuminated by a point 51. Fig. 4b shows a variant with an even lower area density, in which only one pixel 23 is illuminated in an 8x8 pixel matrix.

[0030] According to the invention, only the fully illuminated pixel 23, i.e., the pixel with the highest amplitude value, is evaluated for distance determination, while pixels 23 with lower amplitudes are not used for evaluation. This approach has the advantage that only pixels 23 with a high signal-to-noise ratio (S / N) are used for distance determination.

[0031] Preferably, the points 51 in the point pattern 50 are distributed such that only one pixel 23 is illuminated for each evaluated sub-pixel matrix 24. The position of the point 51 within such a sub-pixel matrix 24 is arbitrary. The distribution of the points 51 in the entire point pattern 51 can therefore also be arbitrary, as long as at least one point 51 is detectable within the given sub-pixel matrix 24. Thus, point patterns 50 with uniformly or statistically distributed points 51, as well as spatially encoded point patterns 50, can be realized. It is also conceivable to distribute the points 51 independently of any evaluated sub-pixel matrix 24.

[0032] Compared to conventional homogeneous illumination, the method according to the invention has the advantage that the points 51 of the emitted dot pattern 50 can be emitted with a higher radiance than would be possible with homogeneous illumination using the same energy input. The potential signal gain depends directly proportionally on the density of the points 51 in the dot pattern. At a density of 1:4, i.e., 1 pixel out of 4 pixels is illuminated, the signal gain increases by a factor of 4. At a ratio of 1:10, it increases by a factor of 10, and at 1:1000 by a factor of 1000.

[0033] By concentrating the energy on a few points, the operating range of the time-of-flight camera system can be increased with regard to ambient light resistance and / or distance, at the cost of spatial resolution. With a dot pattern having a density of 1:10, the resolution of the time-of-flight sensor 22 decreases accordingly by a factor of 10, while the signal-to-noise ratio and thus also the ambient light resistance increase by a factor of 10.

[0034] If the light is also emitted in a coded dot pattern 50, then in addition to TOF distance measurement, distance determination via triangulation, as is known from so-called structured light methods, can also be used.

[0035] Starting from the known distance between the light source 12 of the illumination 10 and the time-of-flight sensor 22 of the camera 20, the distance of the detected point 51 can be determined from the spatial displacement of the detected points 51 of the point pattern 50.

[0036] Fig. Figure 5 shows a possible cross-section of the amplitudes A B in the light distribution of a dot pattern 50 emitted by the illumination 10. In the present case, the light points 51 are not equidistantly distributed. The width of the amplitude peaks of the light points 51 is preferably selected such that, upon reception of these light points 51, essentially only one light-time-of-flight pixel 23 is illuminated.

[0037] Fig. 6 a cross-section of a light distribution in which, in addition to the dot pattern 51, a light with a base amplitude A BB is emitted. In contrast to the light distribution according to Fig. 5. The light emission between the peaks does not fall to zero, but only to the basic amplitude A. BB The amplitude or brightness of the emitted light decreases as is known with 1 / r. 2 ab. The basic amplitude A BB is chosen such that the light detected by the time-of-flight sensor 22 in the immediate vicinity has a sufficient signal strength compared to the background noise. With increasing distance, the signal-to-noise ratio of this base amplitude A decreases. BB increasingly worse, while the high-amplitude light points still exhibit a sufficient signal-to-noise ratio.

[0038] This approach has the advantage that, in the near range, the scene can be fully illuminated with an amplitude sufficient for distance determination, and the distance can be determined with the full resolution of the time-of-flight sensor 22. In the far range, however, the strength of the base amplitude is no longer sufficient. Distance determination is then carried out based on the light points emitted with high intensity, but with lower resolution.

[0039] Fig. 7 and Fig. Figure 8 shows exemplary possible signal / amplitude curves A S of the detected modulated light via a light time-of-flight sensor cross-section x S for near or far range. According to the invention, it is provided that only signals A S to be evaluated for distance measurement, which has an amplitude limit A SG exceed. In the Fig. In the case shown in Figure 7, this applies to all pixels in the cross-section shown. Fig. Figure 8 shows a case where this is related to the base amplitude A. B The emitted and then received modulated light or useful light by the light time-of-flight sensor 22 no longer exceeds the amplitude limit A SG exceeds. In this case, only the light points 51 of the dot pattern 50 emitted with a larger amplitude exceed the amplitude limit A. SG and can be evaluated.

[0040] In order to obtain the highest possible signal-to-noise ratio, the invention further provides that in Fig. In the case outlined in point 8, only the pixels with the maximum amplitude within a predefined sub-pixel range are evaluated. The distance determined for this pixel is then used as the distance value for the entire sub-pixel range.

[0041] Whether the sensor switches to sub-pixel mode can be determined, for example, by the number of pixels that define the amplitude limit A. SGexceeding or falling below the amplitude limit. For example, switching to sub-pixel mode could occur if more than 50% of the pixels exceed the amplitude limit A. SG must not be exceeded. Of course, other limits can be set depending on the application.

[0042] Furthermore, it is also conceivable, as in Fig. Figure 9 shows how to create dot patterns with different amplitudes. Besides the light with base amplitude A. BB A first dot pattern with a first peak amplitude A will be generated. BP1 and a second dot pattern with a second larger peak amplitude A BP2 emitted. If the first dot pattern can no longer be meaningfully detected, the second dot pattern with the larger amplitude A remains. BP2Since this dot pattern will typically have a lower dot density, the sensor can also be divided into larger sub-pixel areas and evaluated in the aforementioned manner.

[0043] To determine the signal amplitude at the sensor, the simplest approach would be to determine the sum of the charge accumulated at both integration nodes within the integration interval. However, this method has the disadvantage that the background light also contributes to the amplitude.

[0044] According to the invention, it is therefore provided that a useful light amplitude is derived from the charge differences q. a - q b or to determine the equivalent voltage differences for two different phase measurements, according to A=(qa−qb)φ02+(qa−qb)φ122 with q a , b = Charges at the integration nodes Ga, Gb and with φ 0,1= Phase position of the respective measurement, for example with φ0 = 0° and φ1 = 90°. This approach has the advantage that, by calculating the difference, the ambient or background light is virtually eliminated, and the useful light amplitude is therefore fundamentally dependent only on the modulated emitted and received useful light.

[0045] The example given here of using two phase measurements to determine amplitude serves only to illustrate and distinguish it from using the sum signal. The method is equally transferable and applicable to other modulation techniques such as n-phase modulation, multi-frequency modulation, pn-modulation, etc. Reference sign 1 Light time-of-flight camera system 10 Lighting modules 12 Lighting 20 receivers, time-of-flight camera 22 Light time-of-flight sensor 23 light-time-time pixels 25 Optics 30 Modulator 35 Phase shifters, lighting phase shifters 38 Modulation control unit 40 objects 50-dot pattern 51 point, light point φ, Δφ(t L ) runtime-related phase shift φ var Phase position φ0 basic phase M0 modulation signal p1 first phase p2 second phase Sp1 transmit signal with first phase SP2 receive signal with second phase Ga, Gb Integration node d object distance q charge

Claims

[1] Time-of-flight camera system (1) with an illumination (10) for emitting modulated light (Sp1) with a predetermined wavelength and a time-of-flight camera (20) with a time-of-flight sensor (22) having several time-of-flight pixels (23), which are configured as PMD pixels with modulation gates (Gam, Gbm) and at least two integration nodes (Ga, Gb), with an evaluation unit designed in such a way that a distance value (d) is determined on the basis of a phase shift (Δφ) between the emitted (Sp1) and a received light (Sp2), where, to determine the phase shift (Δφ), a difference in the charges applied to the integration nodes (Ga, Gb) after an integration time or their voltage equivalent is formed, characterized by , that the camera (20) has an edge or bandpass filter which is transparent to the specified wavelength of the emitted light, and that the illumination (10) is designed such that, in addition to emitting modulated light with a basic amplitude (A BB ) a modulated light in the form of a dot pattern (50) is emitted, the dots (51) of which have an amplitude (A B ) exhibit a value greater than the baseline amplitude (A BB ), where light emission between the points of the dot pattern (50) is not reduced to zero, but only to the basic amplitude (A) BB ) falls off, where the number of points (51) in the dot pattern (50) is less than the number of light time-of-flight pixels (23) of the light time-of-flight sensor (22), and the evaluation unit is designed such that only the pixels (23) of the time-of-flight sensor (22) whose detected useful light amplitude (A) are used to determine a distance value S ) a given amplitude limit (A SG exceed, where the amplitude limit (A SG ) or / and the basic amplitude (A BB ) are defined such that in a near range the basic amplitude (A BB ) the amplitude limit (A SG ) exceeds and falls below in a distant range. [2] Time-of-flight camera system (1) according to claim 1, wherein the evaluation unit is configured such that the time-of-flight sensor (22) is divided into several sub-pixel areas when the amplitude limit (A) SG ) is no longer reached by a minimum number of pixels (23), where only the light-time-of-flight pixel (23) is used in this sub-pixel area to determine a distance value (d, φ) which has a maximum useful light amplitude (A) in this sub-pixel area. S ) exhibits. [3] Time-of-flight camera system (1) according to any of the preceding claims, wherein the ratio of the number of emitted light points (51) to the number of time-of-flight pixels (23) of the time-of-flight sensor (22) is 1:4 or less.

Images