Light time-of-flight camera

Abstract

Time-of-flight camera operating on a phase measurement principle, with illumination for emitting modulated light, with a receiver for receiving the emitted light and the light reflected from a scene, wherein the receiver has at least one time-of-flight pixel designed as a photomixing detector, wherein at least one light-time-of-flight pixel has at least two integration nodes for the accumulation of photogenerated charges, wherein light with different modulation frequencies (f1, f2) is emitted and received in at least two measurement cycles for distance determination and phasers (z1, z2) are determined in each case, whereby the evaluation unit is designed in such a way that that a combined phasor (Z) is generated from the data of a first and second measurement cycle c ) is formed according to the following rule: zc = ( ( 1 − α ) z 1 N 1 + α z 2 N 2 ) z 1 u − N 1 z 2 v where the variables u, v are determined using an extended Euclidean algorithm 1 u,v = egcd(f1,f2) and the coefficient (α) is determined according to the rule α = 1 1 2 − v N 1 be formed where z1 and z2 are the phasors of the first and second measurement cycles and N1, N2 are the least common multiples of a first and second modulation frequency (f1, f2).

Description

[0001] A time-of-flight camera operating on the phase measurement principle is proposed, according to the category of independent claim.

[0002] From DE 10 2013 207 647 A1 a method for distance measurement according to the phase measurement principle is already known, in which several modulation frequencies are used to extend the uniqueness range.

[0003] From DE 10 2021 103 581 A1 a method for correcting depth images of a time-of-flight camera is known, wherein in at least two phase measurements light with different modulation frequencies is emitted, wherein at least two frequency factors are determined according to n1f2 = n2f1 to calculate a distance value.

[0004] The object of the invention is to make a time-of-flight camera more robust, especially against multipath propagation.

[0005] The problem is solved by the time-of-flight camera according to the invention.

[0006] By formulating a new algorithm, the unwrapping problem in Time-of-Flight (ToF) processing of multi-frequency methods becomes fully continuous and differentiable. This differentiability enables better training of neural networks and the application of optimization methods, as well as implementation as an analog circuit.

[0007] The raw images from a time-of-flight camera are measurements of phase-shifted periodic functions. The phase is proportional to the depth d and can be determined for each pixel. Typically, the raw data are sinusoidal signals, which are referred to as the real (Re) and imaginary (Im) parts. Re=−A sin(ϕ) Im=A cos(ϕ) A=Re2+Im2

[0008] The amplitude was denoted by A. This results in the phase ϕ being: ϕ=atan2(−Re,Im)

[0009] The depth is therefore: d=ur2π

[0010] The uniqueness range (ur) of a ToF camera limits the maximum range and is inversely proportional to the modulation frequency: ur=c2f

[0011] The speed of light is denoted by c.

[0012] The dual-frequency method is commonly used to increase the uniqueness range. By using two modulation frequencies f1 and f2, the number of raw images is doubled, resulting in two depths d1 and d2 with uniqueness ranges ur1 and ur2. The noise-resistant combination of both depths into a single depth is achieved using the Chinese remainder theorem: dc=urc2(N1d1ur1+N2d2ur2+M(2 v N1−1)) M=round(N2d2ur2−N1d1ur1)

[0013] Here, N1 and N2 are the least common multiples of f1 and f2. The Bezout coefficient v can be determined using the extended Euclidean algorithm. The extended uniqueness domain is then... cnamed and calculated as follows: Because M is not differentiable at all points, formula 1 cannot be used (or can only be used poorly) in gradient-based optimization methods. If the rounding function is replaced by a linear function to make the algorithm differentiable, it is no longer robust against noise.

[0014] The problem of the lack of differentiability can be solved by unwrapping directly on the raw data. First, we simplify the calculation by converting the raw data into the phases z1 and z2: Re1=−A1 sin(ϕ1) Im1=A1 cos(ϕ1) Im2=−A2 sin(ϕ2) Im2=A1cos(ϕ2) z1=Im1−i Re1=A1 exp (i ϕ1) z2=Im2−i Re2=A2 exp (i ϕ2)

[0015] The combined phasor is obtained by a weighted average of two phasors with compatible frequencies: zc=((1−α)z1N1+α z2N2)z1u−N1z2vu,v=egcd(f1,f2)

[0016] The noise behavior of z c This depends firstly on the noise level of measurements z1 and z2 and secondly on the choice of parameter α. With the following definition of α, correlated components in the noise cancel each other out (instead of reinforcing each other). α=112−v N1

[0017] The extended Euclidean algorithm is denoted by egcd and yields the Bezout coefficients u and v. It is easy to see that z c The distance measurement is differentiable at all points. The distance measurement is calculated as follows: dc=urc2πatan2(Real(zc),Imag(zc))

[0018] Thus, this unwrapping method is defined in a fully differentiable way. A significant advantage of this invention is therefore that gradient-based optimization methods can be applied from the raw data to the combined phasor.

[0019] Another advantage is the simple implementation of this method using an analog circuit, for example, by combining analog multipliers with log and antilog amplifiers. This allows an analog circuit to be implemented either as a separate integrated circuit in a time-of-flight camera or directly within a 4-tap photomixing detector pixel to acquire a phasor with an extended measurement range in a single measurement. Alternatively, the processing can be performed in a digital integrated circuit.

[0020] The method can be extended to more than two frequencies, analogous to the Chinese remainder theorem, by mapping the N-frequency problem, for example, to the calculation of multiple frequency pairs.

[0021] Additionally: Unwrapping of the raw data to a combined phasor via digital integrated circuit on-device or on-chip.

[0022] Preferably, an evaluation unit is designed such that a combined phasor according to the invention is formed from the recorded data.

Claims

[1] Time-of-flight camera operating on a phase measurement principle, with illumination for emitting modulated light, with a receiver for receiving the emitted light and the light reflected from a scene, wherein the receiver has at least one time-of-flight pixel designed as a photomixing detector, wherein at least one light-time-of-flight pixel has at least two integration nodes for the accumulation of photogenerated charges, wherein light with different modulation frequencies (f1, f2) is emitted and received in at least two measurement cycles for distance determination and phasers (z1, z2) are determined in each case, whereby the evaluation unit is designed in such a way that that a combined phasor (Z) is generated from the data of a first and second measurement cycle c ) is formed according to the following rule: zc=((1−α)z1N1+α z2N2)z1u−N1z2v where the variables u, v are determined using an extended Euclidean algorithm 1 u,v = egcd(f1,f2) and the coefficient (α) is determined according to the rule α=112−v N1 be formed where z1 and z2 are the phasors of the first and second measurement cycles and N1, N2 are the least common multiples of a first and second modulation frequency (f1, f2). [2] Time-of-flight camera according to claim 1, wherein the combined phasor (Z) c ) a distance value is determined.