Light time-of-flight camera

Abstract

Time-of-flight camera designed for distance measurement using a phase measurement principle, - with a lighting system for emitting modulated light, - and with a time-of-flight sensor, with at least one time-of-flight pixel, for receiving the emitted and reflected modulated light from a scene, wherein the light-time-of-flight pixels have at least two modulation gates (Gam, Gbm) arranged in a light-sensitive area and at least two integration notes (Ga, Gb), - where at least two integration intervals are provided for distance measurement, wherein within a respective integration interval the modulation gates (Gam, Gbm) are sequentially driven with a first and second modulation signal (CW, UR) and charges from the photosensitive area are accumulated at the integration nodes (Ga, Gb), where the first and second modulation signals (CW, UR) differ in their modulation frequencies, where the CW modulation frequency (f1) is an integer multiple n of the UR modulation frequency (f2) with f1 = n * f2, where n CW periods form a CW clock sequence, wherein during a CW clock sequence (CW) and during UR modulation (UR) either only 1 light pulse or n - 1 light pulses are emitted, where, after an integration interval, the accumulated charges or a measured quantity representing the charge are read out at the integration nodes (Ga, Gb). where in the second integration interval the first modulation signal (CW) is chosen to be complementary and the second modulation signal (UR) is chosen to be identical to the first interval. where a relative distance is obtained from a difference A - B of the correlations determined in the first and second integration intervals, and a uniqueness range is determined from a sum (A + B) of the determined correlations.

Description

[0001] The invention relates to a time-of-flight camera according to claim 1.

[0002] Time-of-flight cameras or time-of-flight camera systems refer in particular to all time-of-flight or 3D-TOF camera systems that derive time-of-flight information from the phase shift of emitted and received radiation. PMD cameras with photomixing detectors (PMDs), such as those described in DE 197 04 496 A1, are particularly suitable as time-of-flight or 3D-TOF cameras.

[0003] German patent DE 10 2021 113 743 A1 relates to a method and a device for measuring the time-of-flight (ToF) of a scene. A ToF sensor performs several measurements with a first modulation frequency to obtain initial readings. The correlation function of these measurements is periodic and shows an increasing amplitude over distance. These readings are used to determine the distance to an object in the scene. The device also includes a processing circuit that calculates this distance. Problems such as glare and pixel saturation due to strong reflections are reduced by the periodic correlation function. Additionally, a second modulation frequency can be used to improve measurement accuracy. The device can be used in applications such as long-range ToF sensing in vehicles.

[0004] German patent DE 10 2022 102 992 A1 relates to a time-of-flight distance measurement system, in particular a time-of-flight camera system, comprising an illumination unit for emitting and a time-of-flight sensor for receiving and demodulating modulated light. A modulator generates a modulation signal for both components. In the distance measurement, a pseudo-noise signal with a first modulation frequency is used to select a distance range. In the control measurement, a common-mode signal with a second, lower modulation frequency is provided. The control measurement compares the amplitude from the distance measurement with that from the control measurement; if the result is outside the tolerated range, the distance measurement is discarded. The substitution of bits of the base-PN sequence by sub-bit sequences enables a symmetrical charge carrier distribution.The use of maximum sequences improves autocorrelation properties and reduces errors due to stray light and multipath interference. The invention offers precise distance measurements and can reliably detect objects within a selected distance range, while objects outside this range are ignored.

[0005] German patent DE 10 2021 117 139 A1 relates to a time-of-flight camera system and a method for its operation. It comprises an illumination unit and a time-of-flight sensor with time-of-flight pixels that have integration nodes for accumulating photogenerated charges. For distance determination, the illumination and sensor are subjected to a modulation signal at integration intervals, and the distance is determined based on the charge differences at the integration nodes. The system performs two measurements: a common-mode (CM) measurement with different coded modulation signals and a continuous-wave (CW) measurement with different phase angles of a periodic CW modulation signal. The distance value is determined from the charge differences of both measurements. The CM measurement allows for a rough determination of the distance range, while the CW measurement improves the distance resolution.The CM measurement divides the distance range into sectors, which are determined based on charge differences or their binary representation. The combination of both measurements allows for an accurate and stable distance measurement, with the CM measurement handling the sector assignment and the CW measurement determining the precise position within the sector.

[0006] US Patent 2020 / 0301014 A1 discloses a method and system for performing depth measurements to resolve distance ambiguities. The method involves performing an indirect time-of-flight (ToF) measurement for each pixel to obtain a value for the apparent distance to an object. Additionally, a first and a second coded modulation measurement are performed to obtain a first and second correlation value, respectively. These measurements use specific combinations of modulation code and reference signal such that the resulting correlation peaks each cover a first and a second, contiguous and overlapping distance range. By comparing the correlation values ​​with threshold values, first and second mask values ​​are determined.These mask values ​​are used to determine whether the apparent distance corresponds to an actual distance within the first or second distance range, thus resolving the ambiguity.

[0007] German patent application DE 10 2020 215 041 A1 discloses a LiDAR sensor system comprising a transmitter and a receiver. The transmitter is designed to emit multiple light transmission sequences, each sequence having a phase code applied by a phase modulator. An evaluation unit is designed to generate an evaluation signal for a multitude of predefined code shifts by multiplying a received light signal by the phase code shifted by the respective code shift. A Doppler frequency is determined from the generated evaluation signals, in particular their spectra. Based on this Doppler frequency and the corresponding code shift of the evaluation signal containing the Doppler frequency, the distance to an object is determined.

[0008] US patent 2020 / 0167942 A1 discloses a method for performing depth measurements with an image sensor to eliminate phase ambiguities. The method includes performing one or more continuous phase measurements and a coded modulation measurement for at least one pixel. Based on the coded modulation measurement, a mask value for the pixel is determined by comparing the measurement to a threshold value. This mask value is then applied to a distance value calculated from the continuous phase measurements. This results in a masked distance value for the pixel that is free of phase-induced ambiguity and thus ensures an unambiguous distance measurement within a defined range.

[0009] The following is a more detailed explanation of some of the terms used: - Continuous-wave or CW measurement refers to amplitude-modulated continuous light emission, whereby a phase difference between the emitted and received modulated light is determined in a CW measurement. - Code modulation or CM measurement uses coded modulation or so-called pseudo-noise modulation instead of uniform modulation. The unambiguous range (UR) describes the maximum range for which a unique distance can be determined in a measurement based on the phase measurement principle. For distances greater than half the wavelength of the modulation signal, the measurement is no longer unambiguous. The unambiguous range can be increased by performing measurements with several different modulation frequencies. - dToF: direct Time of Flight method, all distance measurement methods that determine a distance directly from the 'flight time' of emitted light - iToF: indirect Time of Flight method, all methods that, unlike dToF methods, determine distance using indirect parameters dependent on light travel time, such as a phase shift between a transmitted and received modulation signal.

[0010] Achieving the most accurate and long-range distance measurement possible with iToF systems presents a particular challenge, as the distance noise is linear, while the uniqueness range size is inversely proportional to the modulation frequency of a continuous-wave measurement.

[0011] This discrepancy is typically resolved by cleverly combining several consecutive individual measurements. For this purpose, a CW measurement is combined either with another CW measurement or a coded modulation (CM) measurement. In the first case, the uniqueness range of the combined measurement is increased by a factor of typically 4 to 8 by selecting the uniqueness ranges of the individual measurements according to the principle of the Chinese remainder theorem. In the second case, the position of the uniqueness range of the CW measurement is determined by the CM measurement, thus multiplying the size of the combined uniqueness range.

[0012] Both methods have different advantages and disadvantages. A common disadvantage of both methods is the need for additional measurements, which require additional measurement time, data readout, and electrical energy. In many cases, this is also associated with limitations regarding laser safety due to the additional active illumination of the measurement scene.

[0013] The purpose of the invention is to overcome this disadvantage.

[0014] The problem is solved by the time-of-flight camera according to claim 1.

[0015] The effective uniqueness range of a continuous-wave measurement according to the iToF principle can thus be multiplied by additional code modulation measurements while maintaining the distance noise.

[0016] They show: Fig. 1 schematically a time-of-flight camera system, Fig. 2 a modulated integration of generated charge carriers, Fig. 3 a cross-section through a PMD time-of-flight pixel with a potential distribution Fig. 4. Charge integration patterns depending on the phase shift and position, Fig. 5 a relation of the phase shift in an IQ diagram, Fig. 6 a modulation scheme of a sequential UR and phase determination with one light pulse per UR period, Fig. 7 a modulation scheme of a sequential UR and phase determination with n-1 light pulses per UR period, Fig. 8 Correlation functions of modulation according to Fig. 7,

[0017] Fig. Figure 1 shows a measurement situation for an optical distance measurement with a time-of-flight camera, as is known, for example, from DE 197 04 496 A1.

[0018] The time-of-flight camera system 1 comprises a transmitter unit or illumination module 10 with an illumination 12 and an associated beam shaping optics 15, as well as a receiver unit or time-of-flight camera 20 with a receiving optics 25 and a time-of-flight sensor 22.

[0019] The time-of-flight sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is in particular designed as a PMD sensor. The receiving optics 25 typically consist of several optical elements to improve the imaging properties. The beam-shaping optics 15 of the transmitting unit 10 can, for example, be designed as a reflector or lens optics. In a very simple embodiment, optical elements can optionally be omitted on both the receiving and transmitting sides.

[0020] The measurement principle of this arrangement is essentially based on the fact that, starting from the phase shift of the emitted and received light, the travel time and thus the distance traveled by the received light can be determined. For this purpose, the light source 12 and the light travel time sensor 22 are connected via a modulator 30 together with a specific modulation signal M o with a basic phase position φ0. In the example shown, a phase shifter 35 is also provided between the modulator 30 and the light source 12, with which the basic phase φ0 of the modulation signal M0 of the light source 12 is shifted by defined phase positions φ var can be shifted. For typical phase measurements, phase positions of φ are preferably used. var = 0°, 90°, 180°, 270° used.

[0021] Depending on the set modulation signal, the light source 12 sends an intensity-modulated signal S p1with the first phase position p1 or p1 = φ0 + φ var off. This signal S p1 or, in the case shown, the electromagnetic radiation is reflected by an object 40 and arrives with a corresponding phase shift Δφ(t) due to the distance traveled. L ) with a second phase position p2 = φ0 + φ var + Δφ(t L ) as a received signal S p2 on the light time-of-flight sensor 22. In the light time-of-flight sensor 22, the modulation signal M is o with the received signal S p2 mixed, whereby the phase shift or the object distance d is determined from the resulting signal.

[0022] Furthermore, the system has a modulation control unit 27 which, depending on the measurement task at hand, adjusts the phase angle φ. var the modulation signal M0 is changed and / or the modulation frequency is set via a frequency oscillator 38.

[0023] Infrared LEDs are preferably suitable as the illumination source or light source 12. Of course, other radiation sources in other frequency ranges are also conceivable, in particular light sources in the visible frequency range.

[0024] The basic principle of phase measurement is schematically represented in Fig. Figure 2 shows the time course of the modulation signal M0, which controls the lighting 12 and the light-time-of-flight sensor 22. The light reflected from object 40 arrives as the received signal S. p2 according to its light travel time t L phase-shifted Δφ(t L) to the time-of-flight sensor 22. The time-of-flight sensor 22 collects the photonically generated charges q over several modulation periods in the phase of the modulation signal M0 in a first accumulation gate Ga and in a phase shifted by 180° M0 + 180° in a second accumulation gate Gb. From the ratio of the charges qa, qb collected in the first and second gates Ga, Gb, the phase shift Δφ(t) can be determined. L ) and thus determine a distance d of the object.

[0025] Fig.Figure 3 shows a cross-section through a pixel of a photomixing detector such as that known from DE 197 04 496 A1. The modulation photogates Gam, G0, Gbm form the light-sensitive area of ​​a PMD pixel. Depending on the voltage applied to the modulation gates Gam, G0, Gbm, the photonically generated charges q are directed either to one or the other accumulation gate or integration node Ga, Gb. The integration nodes can be configured as gates or diodes.

[0026] Fig. Figure 3b shows a potential profile in which the charges q flow towards the first integration node Ga, while the potential according to Fig.3c allows the charge q to flow towards the second integration node Gb. The potentials are set according to the applied modulation signals. Depending on the application, the modulation frequencies are preferably in the range of 1 to 100 MHz. With a modulation frequency of, for example, 1 MHz, the period is one microsecond, so the modulation potential changes accordingly every 500 nanoseconds.

[0027] In Fig.Figure 3a further shows a readout unit 400, which may optionally already be part of a PMD time-of-flight sensor designed as a CMOS. The integration nodes Ga, Gb, configured as capacitors or diodes, integrate the photonically generated charges over a multitude of modulation periods. The voltage then applied to the gates Ga, Gb can be tapped off, for example, via the readout unit 400, using a high-impedance connection, as is known. The integration times are preferably selected such that the time-of-flight sensor or the integration nodes and / or the light-sensitive areas do not reach saturation for the expected amount of light. The readout paths of the two integration nodes Ga, Gb can also be referred to as A and B channels.

[0028] Fig. 4a and Fig. Figure 4b shows the curves of the charge difference Δq = q a - q b / (q a + q b ) as a function of the phase shift Δφ(tL ) of the received light signal S p2 with different phase positions. The Fig. Figure 4a shows a curve for an unshifted modulation phase M0 with a phase position φ var = 0°.

[0029] Upon arrival of signal S p2 without phase shift, i.e., Δφ(t L ) = 0°, for example, when the transmitted signal S p1 The phases of the modulation M0 and of the received signal S are directed directly onto the sensor. p2 identical, so that all generated charge carriers are detected phase-synchronously at the first gate Ga and thus a maximum difference signal with Δq = 1 is present.

[0030] With increasing phase shift, the charge at the first accumulation gate Ga decreases and at the second accumulation gate Gb increases. For a phase shift of Δφ(t LAt 90°, the charge carriers qa and qb are equally distributed at both gates Ga and Gb, and the difference is therefore zero, and after a 180° phase shift, it is "-1". With a further increasing phase shift, the charge at the first gate Ga increases again, so that the charge difference ultimately rises again, reaching a maximum at 360° and 0°, respectively.

[0031] Mathematically, this is a correlation function of the received signal S. p2 with the modulating signal M0. q(τ)=∫0τSp2(t−τ)M0(t)dt

[0032] As previously shown, modulation with a square wave results in a triangular correlation function. Modulation with, for example, a sine wave would result in a cosine function.

[0033] How Fig. Figure 4a shows that a measurement of the phase with a phase position is only possible up to a phase shift Δφ(t L ) ≤ 180° unambiguously.

[0034] For maximum detection of the phase shift, the IQ (In-phase quadrature) method is known, in which two measurements are carried out with phase positions shifted by 90°, for example with the phase position φ. var = 0° and φ var = 90°. The result of a measurement with the phase angle φ var = 90° is in Fig. 4b shown.

[0035] The relationship between these two curves can be described in a known way, for example for sinusoidal curves in an IQ diagram according to... Fig. 5. As a first approximation, this representation can also be readily applied to the triangular functions shown.

[0036] The phase angle can then be determined in a known manner using an arctan function: φ=arctanΔq(90°)Δq(0°)

[0037] To compensate for sensor asymmetry, for example, additional phase measurements shifted by 180° can be performed, so that the phase angle can be determined as follows. ϕ=arctan2Δ(90°)−Δq(270°)Δ(0°)−Δq(180°)

[0038] From the propagation-related phase shift Δφ(t L For object distances d that are smaller than half the wavelength λ of the modulation frequency d ≤ λ / 2, a distance can be determined in a known manner. d=Δφ(tL)λ2π⋅12

[0039] For distances d > λ / 2, there is usually no way to measure the phase shift absolutely, so that the determined phase shift can no longer be uniquely assigned to a distance value.

[0040] Fig.Figure 6 shows a possible timing for the illumination and the time-of-flight pixels of a time-of-flight sensor. According to the invention, within a single integration interval, during which charges accumulate at the integration nodes Ga and Gb, the time-of-flight sensor and the time-of-flight pixels are operated sequentially with a first and second modulation signal, CW and UR, respectively, wherein the modulation signals have different frequencies. In the illustrated example, the first modulation signal, CW, has a frequency four times higher than the second modulation signal, UR. The CW modulation frequency (f1) is an integer multiple n of the UR modulation frequency (f2) with f1 = n * f2, where n CW periods form a CW clock sequence, and during a CW clock sequence (CW) and during the UR modulation (UR), either only one light pulse or n - 1 light pulses are emitted.

[0041] Fig.Figure 7 shows a variant in which one light pulse is omitted.

[0042] Typically, for symmetry reasons, a further measurement in a second integration interval with a complementary phase shift of 180° is provided in a 4-tap measurement. According to the invention, however, only the high-frequency modulation signal, here CW, is shifted by 180°, while the low-frequency modulation signal, here UR, is not shifted in phase.

[0043] If necessary, further measurements (3rd and 4th integration intervals) can be performed with orthogonal phase shifts, as well as phase shifts of 90° and 270°. In the orthogonal phase measurement, both the first and second modulation signals, CW and UR, are shifted by 90°. In the complementary measurement at 270°, only the higher-frequency modulation signal, here CW, is shifted, while the lower-frequency modulation signal, here UR, remains at its original 90° phase shift.

[0044] Fig. Figure 8 shows two correlation functions as examples, which result from a control according to Fig. 7 with 0° (A) and 180° (B) phase angle and a selection according to the invention.

[0045] From the differences A - B of the correlation values, the two integration intervals are used to determine a distance and from the sum of the correlation values ​​a uniqueness range.

[0046] With four integration intervals, the evaluation could proceed as follows, for example: In the sum signal A+B, the higher-frequency modulated components cancel each other out, allowing conclusions to be drawn about the position of the CW uniqueness range 1 or 3, or 2 or 4. For this purpose, the sign of the sum signal is evaluated. In the example from Fig. 8. A positive sign indicates a uniqueness range of 1 or 4, a negative sign indicates a uniqueness range of 2 or 3. For the orthogonal measurement, a positive sign indicates a uniqueness range of 1 or 2, and a negative sign indicates a uniqueness range of 3 or 4. The combination of the two measurements thus allows for the unambiguous determination of the uniqueness range. Signal / CW Uniqueness Range UR 1 UR 2 UR 3 UR 4 Sum signal A+Baus 1st and 2nd integration interval + - - + Sum signal A+Baus 3rd and 4th integration interval orthogonal to 1st and 2nd. + + - -

[0047] The ratio of low-frequency to high-frequency components in the combined modulation signal should preferably be chosen such that, for a given scene, the A / B analysis of the low-frequency components remains unambiguous, while the high-frequency component should be maximized to achieve lower phase noise. This also maximizes the symmetry of the 0° and 180° (or 90° and 270°) measurements, thus reducing systematic measurement and readout errors in individual measurements by means of difference calculation. The choice of the low- to high-frequency modulation ratio can be either statically determined a priori or adaptively during operation.

Claims

[1] Time-of-flight camera designed for distance measurement using a phase measurement principle, - with a lighting system for emitting modulated light, - and with a time-of-flight sensor, with at least one time-of-flight pixel, for receiving the emitted and reflected modulated light from a scene, wherein the light-time-of-flight pixels have at least two modulation gates (Gam, Gbm) arranged in a light-sensitive area and at least two integration notes (Ga, Gb), - where at least two integration intervals are provided for distance measurement, wherein within a respective integration interval the modulation gates (Gam, Gbm) are sequentially driven with a first and second modulation signal (CW, UR) and charges from the photosensitive area are accumulated at the integration nodes (Ga, Gb), where the first and second modulation signals (CW, UR) differ in their modulation frequencies, where the CW modulation frequency (f1) is an integer multiple n of the UR modulation frequency (f2) with f1 = n * f2, where n CW periods form a CW clock sequence, wherein during a CW clock sequence (CW) and during UR modulation (UR) either only 1 light pulse or n - 1 light pulses are emitted, where, after an integration interval, the accumulated charges or a measured quantity representing the charge are read out at the integration nodes (Ga, Gb). where in the second integration interval the first modulation signal (CW) is chosen to be complementary and the second modulation signal (UR) is chosen to be identical to the first interval. where a relative distance is obtained from a difference A - B of the correlations determined in the first and second integration intervals, and a uniqueness range is determined from a sum (A + B) of the determined correlations. [2] Time-of-flight camera according to claim 1, wherein a modulation signal is designed as a coded modulation (CM), in particular as a Gray code. [3] Time-of-flight camera according to one of the preceding claims, wherein the ratio of the high- and low-frequency modulation components is adjustable depending on a measurement task. [4] Time-of-flight camera according to one of the preceding claims, wherein the ratio of the high- and low-frequency modulation components is determined prior to a measurement task. [5] Time-of-flight camera according to one of the preceding claims, wherein the ratio of the high- and low-frequency modulation components is adaptively determined based on a signal noise determined in a first measurement.

Images