Multipath error compensation method and multipath error compensation indirect flight time distance calculation device

Abstract

The present application discloses a method for multipath error compensation and an indirect time-of-flight distance calculation device for multipath error compensation. A method for multipath error compensation includes illuminating a scene with an unstructured light source (142) during a plurality of consecutive time frames. A structured light source (144) illuminates the scene simultaneously with the unstructured light source illuminating the scene, and the illumination of the structured light source (144) occurs during a predetermined frame of the plurality of consecutive time frames. A photodetector array (102) generates a plurality of sets of signals in response to illumination by light reflected from the scene according to an indirect time-of-flight measurement technique. An error estimate (130) is derived from the plurality of sets of signals generated during a selected time frame of the plurality of consecutive time frames when structured illumination occurs and another time frame temporally related to the selected time frame when structured illumination and unstructured illumination occur. The error estimate is applied to a distance calculation about the scene.

Description

Technical Field

[0001] The present invention relates to a method for multipath error compensation of an indirect time-of-flight distance calculation device, the method being of the type that utilizes structured light to illuminate a scene. The present invention also relates to a multipath error compensated indirect time-of-flight distance calculation device, the device being of the type that includes a structured light source to illuminate a scene. Background Art

[0002] In so-called time-of-flight sensing systems and other systems (such as game console vision systems), it is known to employ an illumination source to illuminate the surrounding environment (sometimes referred to as a "scene") within the field of view of the illumination source, and to process the light reflected by features of the scene. Such so-called LiDAR (Light Detection and Ranging) systems use an illumination source to illuminate the scene with light, and use a detection device (such as a photodiode array, some optical elements, and a processing unit) to detect the light reflected from objects in the scene. The light reflected from the objects in the scene is received by the detection device and converted into an electrical signal, which is then processed by the processing unit by applying a time-of-flight (ToF) calculation in order to determine the distance of the object from the detection device. Although different kinds of LiDAR systems are known based on different operating principles, such systems basically illuminate the scene and detect the reflected light.

[0003] In this regard, so-called "flash LiDAR" technology is a direct ToF ranging technology that employs a light source that emits light pulses that are subsequently reflected by features of the scene and detected by a detector device. In such technology, the distance to the reflecting feature is calculated directly using the measured time for the light pulse to travel to and from the reflecting feature and return to the detector device. The light pulses incident on the detector device are sampled in the time domain at a very high sampling rate. Therefore, the signal path in the processing circuitry that implements such technology requires a high bandwidth of the signal and a large silicon "footprint", i.e., such implementations require a relatively large area on the silicon wafer, which in turn limits the number of channels that can be supported on the integrated circuit. Therefore, the actual spatial number of channels that such flash LiDAR sensors can support is typically less than 100. In order to overcome this limitation, a mechanical scanning system that requires moving parts is implemented.

[0004] Another known LiDAR system uses the so-called "indirect time of flight" (iToF) ranging technology. The iTOF system emits a continuous wave light signal, and the reflection of the continuous wave light signal is received and analyzed by a detector device. Multiple samples (e.g., four samples) of light reflected from features of the scene are obtained, each sample being phase stepped, for example, 90°. Using this illumination and sampling method, the phase angle between the illumination and the reflection can be determined, and the determined phase angle can be used to determine the distance to the reflective features of the scene. In this regard, a typical iToF system includes a buffer that stores analog signals about m phases for subsequent signal processing generated by a so-called photonic mixer device. A discrete Fourier transform unit calculates the baseband output signal of the complex signal stored by the buffer based on the in-phase component and the orthogonal component of the signal. Using the values ​​of the in-phase component and the orthogonal component, the phase angle and amplitude of the complex signal can be calculated, and the phase angle information can be used to solve the distance to the object.

[0005] In iToF systems, high-frequency signal processing (demodulation) occurs at the pixel level, and therefore the signal bandwidth required to integrate a large number of pixels on the same chip is low post-pixel. Therefore, compared with direct ToF systems, iToF systems can support a larger number of channels and therefore support higher spatial resolution measurements. However, iToF systems have limited distance measurement capabilities. In this regard, in order to achieve low random distance measurement errors, iToF systems require high modulation frequencies, which in turn reduces the range of distances that can be clearly measured. For example, a 100MHz modulation frequency results in an approximate clear measurement range of 1.5m. In addition, conventional iToF systems are susceptible to errors caused by multiple reflections and multiple propagation paths.

[0006] To illustrate this, consider an automotive context, where the scene includes a vehicle being monitored by an iToF system, but the scene also includes a wall and another object, such as a road sign. The distance to the vehicle measured by the iToF system is determined by light reflected directly from the vehicle (i.e., true direct reflection). However, in such scenarios, emitted light may reach the vehicle and / or light reflected by the vehicle may reach the iToF system indirectly via reflections from the wall, resulting in an erroneous measurement of the distance due to the increased optical path length and the iToF system's inability to distinguish the path of the emitted light and / or reflected light. Additionally, signals from highly reflective road signs may be scattered by optical system imperfections, thereby generating a stray light signal received by the iToF system and again resulting in another erroneous measurement of the distance, which is actually the distance to the road sign.

[0007] Another possible source of multipath distance measurement errors is the presence of multiple objects within one pixel of the iToF system's field of view. However, due to the high resolution of iToF cameras, this error is less important in practice. Nevertheless, such sources of error exist.

[0008] US 2015 / 253429 discloses a technique for reducing multipath errors and exploits the fact that multipath errors typically have low spatial frequencies, since they are typically caused by diffuse or semi-diffuse reflections that act as spatial low-pass filters. To compensate for such sources of error, the scene is illuminated by a set of complementary patterns generated using beam shaping optics (e.g., holographic diffuse reflectors, micro-electromechanical system (MEMS) mirror arrays) or simply by masking.

[0009] For example, in an iTOF system using the compensation technique of US 2015 / 253429, frame acquisition consists of separate acquisitions of 2 or more subframes using complementary checkerboard illumination patterns. In each subframe, the illuminated area receives the iTOF illumination light signal from the light area of ​​the checkerboard pattern. The signal includes the real (direct) optical echo signal (ground truth) and multipath components

[0010]

[0011] About the dark areas of the checkerboard lighting pattern The signal consists only of multipath components

[0012]

[0013] This means that on a single frame, for a given dark area The signal can be determined by illumination by a combination of two respective complementary patterns on a first subframe and a second subframe. In order to measure the light reflected from an object in the scene, the first illumination pattern is used to illuminate the scene according to an indirect time-of-flight measurement technique using, for example, 4 offset phase angles and the reflections are measured by the ToF camera on the first subframe. This produces a multipath vector of the dark part of the first illumination pattern Thereafter, the scene is illuminated using a second complementary illumination pattern during a second subframe and the reflections are measured by the ToF camera in a similar manner to the measurements performed on the first subframe. A complete "image" of the vectors, i.e. a matrix of vectors corresponding to each pixel of the ToF camera. Therefore, in this type of method, 8 sampling cycles (e.g. one phase shift applied in each subframe) are required to measure the reflected "ground truth" signal US 2015 / 253429 also discloses other illumination patterns to minimize the number of required sampling cycles.

[0014] The use of complementary illumination patterns allows reconstruction of the illumination light signal over the duration of the entire frame by exploiting the low spatial frequency properties of the multipath error components. Therefore, there is no crosstalk with ToF reflected signals Approximately:

[0015]

[0016] This method performs well enough in practice, but has the disadvantage that as the number of complementary illumination patterns increases, the range frame rate is reduced by a factor of two or less. As a result, the accuracy with which motion in the scene can be identified decreases.

[0017] US 2018 / 095165, US 2013 / 148102, US 10,061,028, US 2019 / 051004, US 2019 / 068853 disclose various techniques for lighting a scene with different light patterns. Summary of the invention

[0018] According to a first aspect of the present invention, there is provided a method for multipath error compensation for an indirect time-of-flight distance calculation device, the method comprising: an unstructured light source illuminating a scene during a plurality of consecutive time frames; a structured light source illuminating the scene simultaneously with the unstructured light source illuminating the scene, the illumination of the structured light source occurring during a predetermined frame of the plurality of consecutive time frames; a photodetector array generating a plurality of groups of signals in response to illumination with light reflected from the scene, the plurality of groups of signals comprising a group of signals substantially simultaneously generated by each photodetector element in the photodiode array according to an indirect time-of-flight measurement technique; deriving an error estimate from the plurality of groups of signals generated during a selected time frame in the plurality of consecutive time frames and another time frame temporally related to the selected time frame, the illumination of the unstructured light source occurring during the selected time frame, and the illumination of both the structured light source and the unstructured light source occurring during the another time frame; and applying the error estimate to a distance calculation regarding the scene.

[0019] The photodiode array may generate the plurality of sets of signals in synchronization with illumination of the scene by the structured light source and / or the unstructured light source received by the photodetector array in each of a plurality of consecutive time frames.

[0020] The method may further include updating the error estimate using the multiple sets of signals generated during subsequent selected frames of the multiple consecutive time frames and further time frames that are temporally related to the subsequent selected time frames, illumination of the unstructured light source occurring during the subsequent selected time frames, and illumination of both the structured light source and the unstructured light source occurring during the further time frames.

[0021] The predetermined times of the plurality of consecutive time frames may be non-consecutive. The non-consecutive predetermined frames are regularly spaced in time.

[0022] The method may further comprise deriving the error estimate by calculating differences between respective measurements corresponding to the plurality of sets of signals for the selected time frame and respective measurements corresponding to the plurality of sets of signals for the another time frame.

[0023] Calculating the difference may include calculating a difference image frame between the image frames of the plurality of groups of signals with respect to the selected time frame and the image frames of the plurality of groups of signals with respect to the another time frame.

[0024] The structured light source may employ an illumination pattern including a non-illuminated area; and the error estimate is derived with respect to the non-illuminated area.

[0025] The error estimate may be derived using the calculated area of ​​the difference corresponding to the non-illuminated area of ​​the illumination pattern of the structured light source.

[0026] The method may further include storing measurements of the difference image frame corresponding to the non-illuminated area.

[0027] The first unstructured light source and the structured light source may share a substantially common illumination field.

[0028] The unstructured light source may emit a uniform illumination pattern. The uniform illumination pattern may be with respect to a scene within a field of view of the photodetector array.

[0029] The calculated difference may include an error measure with respect to the non-illuminated area of ​​the illumination pattern of the structured light source; and the deriving of the error estimate may include: interpolating using the measurement corresponding to the non-illuminated area of ​​the illumination pattern of the structured light source to obtain an error measure with respect to the illuminated area of ​​the illumination pattern of the structured light source.

[0030] The method may further comprise scaling the calculated measure of the difference in relation to the unilluminated area.

[0031] The method may further include: another structured light source illuminating the scene simultaneously with the unstructured light source illuminating the scene; the illumination by the another structured light source may occur during a further predetermined time frame different from the predetermined time frame in the multiple consecutive time frames; and the derivation of the error estimate may further include using the multiple groups of signals generated during another selected time frame of the further predetermined time frame, and the illumination by the another unstructured light source may occur during the other selected time frame.

[0032] The structured light source may include a first illumination pattern. The other structured light source may include a second illumination pattern; the second illumination pattern may be different from the first illumination pattern.

[0033] According to a second aspect of the present invention, there is provided an indirect time-of-flight distance calculation device with multipath error compensation, comprising: an unstructured light source configured to illuminate a scene during a plurality of consecutive time frames; a structured light source configured to illuminate the scene simultaneously with the illumination of the scene by the unstructured light source, the illumination of the structured light source occurring during a predetermined frame of the plurality of consecutive time frames; a photodetector array comprising a plurality of photodetector elements; and a processing resource configured to calculate the distance; wherein the photodiode array is configured to generate a plurality of groups of signals in response to illumination by light reflected from the scene, the plurality of groups of signals comprising a group of signals substantially simultaneously generated by each of the plurality of photodetector elements of the plurality of photodiode elements according to an indirect time-of-flight measurement technique; the processing resource is configured to derive an error estimate from the plurality of groups of signals generated during a selected time frame in the plurality of consecutive time frames and another time frame temporally related to the selected time frame, the illumination of the unstructured light source occurring during the selected time frame, and the illumination of both the structured light source and the unstructured light source occurring during the another time frame; and the processing resource is configured to apply the error estimate to the distance calculation regarding the scene.

[0034] According to a third aspect of the present invention, there is provided a method for multipath error compensation for an indirect time-of-flight distance calculation device, the method comprising: illuminating a scene during a plurality of consecutive time frames using unstructured light; illuminating the scene simultaneously with structured light and unstructured illumination, the structured illumination occurring during a predetermined frame of the plurality of consecutive time frames; generating a plurality of groups of signals according to an indirect time-of-flight measurement technique in response to illumination by light reflected from the scene; deriving error estimates from the plurality of groups of signals generated during a selected time frame among the plurality of consecutive time frames and another time frame temporally related to the selected time frame, the illumination by the unstructured light source occurring during the selected time frame, and the illumination by both the structured light and the unstructured light occurring during the other time frame; and applying the error estimates to distance calculations regarding the scene.

[0035] It is therefore possible to provide an apparatus and method that can avoid or at least mitigate multipath effects in an indirect time-of-flight distance calculation system while mitigating the loss of range frame rate reduction that typically accompanies illumination with different structured light patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] At least one embodiment of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0037] Figure 1 is a schematic diagram of an indirect time-of-flight distance calculation device for multipath error compensation constituting an embodiment of the present invention;

[0038] Figure 2 Another embodiment of the present invention is used for Figure 1 A flowchart of the first part of a method for performing multipath error compensation on an indirect flight time distance calculation device;

[0039] Figure 3 yes Figure 2 A flowchart of the second part of the method; and

[0040] Figure 4 It is a further embodiment of the present invention. Figure 1 Schematic diagram of the lighting arrangement. DETAILED DESCRIPTION

[0041] Throughout the following description, the same reference numerals will be used to identify the same components.

[0042] refer to Figure 1 , the first indirect time-of-flight distance calculation device 100 includes a detection and ranging module, which includes an optical receiver photon mixer pixel device 102 constituting a photodetector element, the optical receiver photon mixer pixel device 102 including a photodiode 104 having an anode operably coupled to a ground potential or other predetermined voltage and a cathode coupled to a first input of a photon mixer 106, the output of which is coupled to an input of an integrator 108. In this example, for simplicity and clarity of description, a single photon mixer pixel device 102 is being described. However, the skilled person will understand that the detection and ranging module includes an array of photon mixer pixel devices of the type described above. The output of the integrator 108 is coupled to an input of a phase buffer 110 having a plurality of parallel outputs representing respective accumulated charge levels with respect to the applied phase values ​​of the photon mixer pixel device 102. In this example, the phase buffer 110 includes m outputs, one output with respect to each applied phase value.

[0043] The skilled person will appreciate that, since the output of the integrator 108 is an accumulated charge, and in this example in the analog domain, the output of the integrator 108 needs to be converted to the digital domain prior to processing downstream of the phase buffer 110. This can be achieved, for example, by employing a photon counter as the integrator 108 or providing an analog-to-digital converter before or after the phase buffer 110.

[0044] The phase signal generator 112 is configured to generate a continuous wave electrical signal. The phase of the continuous wave signal can be selected via the control input 114, and the phase of the continuous wave signal can be selected from the following set of phases: [p1, p2, ..., p m ]. The output of the phase signal generator 112 is coupled to a second input of the photonic mixer 106 .

[0045] The apparatus further comprises a timing control unit 118 having a first control output 120 coupled to the control input 114 of the phase signal generator 112 and a synchronization output 122 operatively coupled to a timing input 124 of the phase buffer 110 .

[0046] The multiple (m) outputs of the phase buffer 110 are respectively coupled to multiple parallel inputs of a digital Fourier transform (DFT) unit 116. The DFT unit 116 has a plurality of digital in-phase (I) / quadrature (Q) outputs corresponding to b pairs of digital I / Q outputs (not shown) for different harmonics of the measured signal.

[0047] A pair of I / Q outputs 126 related to the first harmonic of the received reflected optical signal are coupled to a vector input 128 of a distance calculation unit 130. The distance calculation unit 130 has a multipath calculation control port 132 operably coupled to a second control output 134 of the timing control unit 118. The vector input 128 of the distance calculation unit 130 is operably coupled to a vector buffer 140, which is also operably coupled to a vector to distance calculation unit 136. The vector buffer 140 of the distance calculation unit 130 is also operably coupled to the multipath calculation unit 138. In this example, the vector to distance calculation unit 136 includes a phase angle calculation unit (e.g., an inverse tangent unit for using an arctan2 technique) to calculate the phase angle. However, the skilled person will recognize that other techniques may be used to calculate the phase angle (e.g., trigonometric techniques). Optionally, the vector to distance calculation unit 136 may include an amplitude calculation unit that uses, for example, a taxi norm L1 or a Euclidean norm L2 calculation technique.

[0048] The device 100 also includes a first electromagnetic radiation source (e.g., a laser diode (LD) or a light emitting diode (LED)) (hereinafter referred to as the first light source 142) and a second electromagnetic radiation source (e.g., an LD or an LED) (hereinafter referred to as the second light source 144), which share a substantially common illumination field in this example. In this example, the first light source 142 and the second light source 144 are configured to emit amplitude-modulated infrared light according to an indirect time-of-flight measurement technique so as to be emitted as a first continuous wave optical signal and a second continuous wave optical signal, respectively, when driven. In this aspect, the first driver circuit 146 has a first input operably coupled to a trigger signal output 148 of the timing control unit 118, and the second driver circuit 150 has a second input of an activation controller 152 that is also operably coupled to the trigger signal output 148 of the timing control unit 118, but via, for example, a switching circuit. The activation controller 152 has a control input operably coupled to a source selection output 154 of the timing control unit 118. A first output of the first driver circuit 146 is coupled to a first drive input of the first light source 142, and a second output of the second driver circuit 150 is coupled to a second drive input of the second light source 144. In this example, the first light source 142 is configured to emit a uniform emission of light (i.e., unpatterned light), and the second light source 144 is configured to emit structured light (e.g., a predetermined light pattern), as will now be described in further detail later herein.

[0049] In operation ( Figure 2 and Figure 3 ), after power-on and at the beginning of a measurement time frame (which constitutes a selected time frame of a plurality of consecutive time frames), the timing control unit 118 ensures that the second light source 114 is not activated (i.e., not receiving a trigger signal) and that the first light source 142 is receiving a trigger signal. Therefore, the first light source 142 emits a first uniform continuous wave light signal, which illuminates (step 200) a scene, such as the entirety of the scene. In this aspect, the first light source 142 is an unstructured light source. For example, an object in the scene reflects the emitted first light signal. The first light source 142 illuminates the scene during a plurality of consecutive time frames. During a measurement time frame of a plurality of consecutive time frames, the phase signal generator 112 generates a continuous wave electrical signal, and the timing control unit 118 controls the cyclic traversal of the group of phases with respect to the electrical signal control relative to the continuous wave first light signal. A synchronization signal is also applied by the timing control unit 118 to the frame buffer unit 110 to ensure that a complete signal group is transmitted to the DFT unit 116. The timing control unit 118 also provides a multipath calculation activation signal to the distance calculation unit 130 via the multipath calculation control port 132 to calculate the compensated vectors. This can be achieved, for example, by setting a flag that the multipath calculation unit 138 and the vector to distance calculation unit 136 monitor.

[0050] The electrical signal generated by the phase signal generator 112 is applied to the photonic mixer 106, and the phase angle of the electrical signal cycles through the set of phase angles mentioned above during the measurement time frame, and the frame buffer 110 serially receives the digital representation of the charge stored in the integrator 108 for each of the set of phase angles (steps 202 and 204). In this regard, the integrator 108 provides a serial plurality of phase angle sample outputs representing respective accumulated charge levels for the applied phase angle values ​​of the photonic mixer pixel device 102. Once the phase buffer 110 has cycled through the set of phase angles, it effectively performs a serial-to-parallel conversion of the electrical measurement signal originating from the integrator 108 and then provides m parallel output signals to the DFT unit 116.

[0051] A subsequent measurement time frame constitutes another time frame temporally related to the selected time frame, the subsequent measurement time frame following the measurement time frame, during which, in response to receiving the m output signals, the DFT unit 116 generates (step 206) a pair of I / Q output signals at the I / Q output 126, the I / Q output signals constituting a uniform illumination vector S about the fundamental frequency 均匀 The I / Q output signals are received by the distance calculation unit 130 and temporarily stored in the vector buffer 140 until another pair of I / Q output signals for the subsequent measurement time frame is received.

[0052] Additionally, at the beginning of the subsequent measurement time frame, the timing control unit 118 instructs the activation controller 152 to allow the second driver circuit 150 to receive a trigger signal so as to drive the second light source 144, which is configured to emit a structured light pattern as mentioned above. Figure 4 , the second light source 144 projects a high spatial frequency low intensity illumination pattern, such as a dot pattern generated by a vertical cavity surface emitting laser (VCSEL) array placed in the focal plane of the lens, or a checkerboard pattern. In this aspect, the second light source 144 employs an illumination pattern including a region of no illumination.

[0053] During this subsequent measurement time frame, the structured light pattern is emitted (step 208) simultaneously with the uniform (unstructured) emission of light by the first light source 142 and is reflected by the object. It is therefore concluded that, in general, the second light source 144 illuminates during a predetermined frame of a plurality of consecutive time frames, which in this example are alternating time frames. However, the skilled person will recognize that in other examples, illumination by the second light source 144 may occur after a predetermined number of time frames, and may occur repeatedly if desired.

[0054] When the second light source 144 illuminates the scene, the phase signal generator 112 still generates the continuous wave electrical signal as described above with respect to illumination by only the first light source, and the timing control unit 118 controls the cyclic traversal of the set of phases with respect to the electrical signal relative to the continuous wave first light signal.

[0055] The electrical signal generated by the phase signal generator 112 is again applied to the photonic mixer 106, and during the subsequent measurement time frame, the phase angle of the electrical signal cycles through the set of phase angles mentioned above, and the frame buffer 110 serially receives the digital representation of the charge stored in the integrator 108 for each phase angle in the set of phase angles (steps 210 and 212).

[0056] Once the subsequent measurement time frame is over, the timing control unit 118 prevents (step 214) triggering of the second driver circuit 150, which stops the emission of the second light signal by the second light source 144. The above process of illumination with only uniform light from the first light source 142 followed by combined illumination by the first light source 142 and the second light source 144 is repeated (steps 200 to 214) while calculating the desired associated vectors for the desired time period.

[0057] When another measurement time frame has elapsed, the phase buffer 110 again performs a serial-to-parallel conversion on the electrical measurement signal originating from the integrator 108 and provides m parallel output signals to the DFT unit 116. Using the m parallel output signals, the DFT unit 116 generates (step 216) a subsequent pair of I / Q output signals that constitute a combined illumination vector about the baseband And has a second phase angle. The combined illumination vector The subsequent I / Q output signals of are received by the distance calculation unit 130 and are also stored in the vector buffer 140 .

[0058] The above method is performed for a plurality of pixels including the above mentioned optical receiver photon mixer pixel device 102. In this regard, in order to more clearly understand the operation of the apparatus 100 from this stage of the operating method, the processing related to the uniform illumination vector and the combined lighting vector The vectors are generated with respect to a plurality of pixels (e.g., an array of pixels) including the pixel 102. In this regard, each pixel 102 outputs a plurality of phase angle sample outputs substantially synchronously with the illumination of the scene by the second light source 144 and / or the first light source 142. The plurality of pixels generate a plurality of sets of signals substantially simultaneously, and the DFT unit 116 generates uniform illumination vectors substantially simultaneously at the appropriate time as described above. and the combined lighting vector

[0059] When the measurement time frame elapses, the distance calculation unit 130 operates as follows. In response to subsequent pairs of I / Q output signals sent by the timing control unit 118 to calculate and compensate for multipath errors, the multipath calculation unit 138 accesses the uniform illumination vector for substantially every pixel stored in the vector buffer 140. and the combined lighting vector The structured illumination vector is calculated (step 218) with respect to the second light signal emitted by the second light source 144.

[0060]

[0061] Therefore, in general, it can be seen that the difference image frames are calculated between the image frames of multiple groups of signals regarding a selected time frame (the measurement time frame in this example) and the image frames of multiple groups of signals regarding another time frame (the subsequent measurement time frame in this example).

[0062] Based on the structured illumination vector, the multi-path computation unit 138 uses a priori knowledge of the pattern of structured illumination of the second light signal in order to identify (step 220) non-illuminated areas of the pattern. For those areas corresponding to the non-illuminated areas of the pattern of structured illumination of the second light signal A function f (e.g., a function that performs interpolation) may be used to estimate the light received and reflected due to multipath effects in unlit areas.

[0063]

[0064] In this example, the reflected light about the unilluminated areas of the pattern is estimated by a low pass filtering operation performed by the multipath computation unit 138. This constitutes the error estimate The error estimate is derived using the area corresponding to the non-illuminated area of ​​the illumination pattern of the second light source 144 (the area of ​​the difference calculated in equation (4) above). The measurements of the difference image frame corresponding to the non-illuminated area are stored. Using the vectors corresponding to the identified non-illuminated area of ​​the pattern, the multi-path computation unit 138 uses the function (in this example, the low-pass filter function) to determine the error estimates for all vectors of the difference image frame and then calculates (step 222) the "ground truth" vector The "ground truth" vector represents the measured reflection from the object after compensating for the estimated multipath error:

[0065]

[0066] where k is a scaling constant defined by the optical power ratio between the first light source 142 emitting a uniform light signal and the second light source 144 emitting a structured light signal, and is the multipath error estimate calculated across all vectors of the difference image frame. The ground truth vector or the corrected vector are stored in the vector buffer 140 .

[0067] Error compensation vector Can be applied to the uniform illumination vector and the combined lighting vector After that, the calculated compensated vector (i.e., the reference true value vector ) is accessed by the vector to distance calculation unit 136, and in this example, the inverse tangent unit calculation (step 224) of the vector to distance calculation unit 136 is calculated by the reference truth vector The defined phase angle is then used to calculate the time of flight based on the phase angle calculated using the indirect time of flight measurement technique and a predetermined scaling factor for converting the calculated phase angle to the time of flight (step 226). The distance to the object is calculated based on the calculated time of flight.

[0068] The above steps (steps 216 to 226) are repeated until it is no longer necessary to generate a compensated measured distance. Thus, in this example, the error estimate is updated as needed using multiple sets of signals generated during a selected time frame of a plurality of consecutive time frames after the subsequent measurement time frame, and using multiple sets of signals generated during a further time frame temporally related to the subsequent selected time frame, the illumination of the first light source 142 occurs during the subsequent selected time frame, and the illumination of the first light source 142 and the second light source 144 occurs during the further time frame.

[0069] The skilled person will recognize that the above implementations are merely examples of various implementations that are conceivable within the scope of the appended claims. In fact, it will be appreciated that, for example, in other embodiments, the phase buffer 110 and thereby the parallel processing of the buffered phase measurements serially acquired from the photonic mixer pixel device 102 may be omitted. In such embodiments, the serial transmission of the phase measurements may be directly to the DFT unit 116, thereby reducing the memory requirements of the detection and ranging modules. In this regard, the DFT unit 116 may be provided with a smaller internal buffer to iteratively calculate the intermediate I and Q values ​​of the serially received individual phase measurements for each frame cycle, which are accumulated in the frame cycle to generate the final I and Q value results. To support this arrangement, the DFT unit 116 may be operably coupled to a timing control unit 118 to maintain synchronization of data processing.

[0070] In the above example, the first light source 142 illuminates uniformly over each measurement time frame, and the second light source 144 projects a high spatial frequency illumination pattern every other frame. In other examples, a further light source may be employed to project another pattern (e.g., a pattern complementary to the second light source 144) to further improve multipath error compensation accuracy, the additional pattern being emitted during a different time frame than the illumination projected by the second light source 144, but emitted simultaneously with the uniform illumination of the first light source 142. In this regard, and generally, the illumination of the further light source may occur during a further predetermined frame of the plurality of consecutive time frames that is different from the predetermined frame mentioned above. In such examples, multiple sets of signals may be generated by the pixels during the further predetermined frames, respectively, and an error estimate may be derived by using the multiple sets of signals generated during another selected time frame of the further predetermined frames, during which the illumination of the further light source occurs. In such examples, it should be appreciated that the structured light pattern emitted by the second light source 144 is different from the further structured light pattern emitted by the further light source.

[0071] The above examples describe generating a plurality of uniform illumination vectors substantially simultaneously during a measurement time frame. The measurement time frame may be understood as a time frame selected from a plurality of consecutive time frames for measurement. A plurality of combined illumination vectors are then generated for subsequent measurement time frames occurring immediately after the selected time frame. However, more generally and as already hinted above, a plurality of combined illumination vectors may be generated with respect to another time frame temporally related to the selected time frame. The illumination by the unstructured light source occurs during a selected time frame, and the illumination by both the structured light source and the unstructured light source occurs during another time frame. In this aspect, the plurality of combined illumination vectors The generation of does not have to be performed immediately before or after the selected time frame, and in some examples, it may be possible to generate multiple combined lighting vectors after the selected time frame. A greater number of time frames may elapse before, and / or may allow for the generation of multiple combined lighting vectors Then generate multiple uniform lighting vectors A greater number of time frames have passed before. It can therefore be seen that the predetermined time frames during which the second light source 144 illuminates the scene may be discontinuous. Furthermore, the discontinuous time frames may be regularly spaced in time.

[0072] It should be appreciated that, unless expressly stated otherwise, references herein to "light" are intended as references to the optical range of the electromagnetic spectrum, for example between about 350 nm and about 2000 nm, such as between about 550 nm and about 1400 nm, or between about 600 nm and about 1000 nm.

[0073] Alternative embodiments of the present invention may be implemented as a computer program product for use with a computer system, such as a series of computer instructions stored on a tangible data recording medium (such as a disk, CD-ROM, ROM or fixed disk) or implemented in a computer data signal, which is transmitted through a tangible medium or a wireless medium (such as microwaves or infrared rays). The series of computer instructions may constitute all or part of the above-mentioned functions, and may also be stored in any volatile or non-volatile memory device, such as a semiconductor device, a magnetic device, an optical device or other memory device.

Claims

1. A method for multipath error compensation of an indirect time-of-flight distance calculation device, the method comprising: The structured light source illuminates the scene and employs a lighting pattern that includes unlit areas; a photodetector array generating a plurality of sets of signals in response to illumination by light reflected from the scene, the plurality of sets of signals including a set of signals substantially simultaneously generated by each photodetector element in the photodetector array in accordance with an indirect time-of-flight measurement technique; an unstructured light source illuminating the scene during a plurality of consecutive time frames; The structured light source illuminates the scene simultaneously with the unstructured light source illuminating the scene, the illumination by the structured light source occurring during a predetermined time frame of the plurality of consecutive time frames; deriving error estimates from the plurality of sets of signals generated during a selected time frame among the plurality of consecutive time frames and another time frame temporally related to the selected time frame, illumination of the unstructured light source occurring during the selected time frame, and illumination of both the structured light source and the unstructured light source occurring during the another time frame; as well as applying the error estimate to a distance calculation with respect to the scene; in The error estimate is derived with respect to the non-illuminated area using a priori knowledge of the illumination pattern.

2. The method of claim 1, further comprising: The error estimate is updated using the multiple sets of signals generated during subsequent selected frames of the multiple consecutive time frames and further time frames that are temporally related to the subsequent selected time frames, the illumination of the unstructured light source occurring during the subsequent selected time frames, and the illumination of both the structured light source and the unstructured light source occurring during the further time frames.

3. The method according to claim 1, characterized in that The predetermined time frames of the plurality of consecutive time frames are non-consecutive.

4. The method according to claim 3, characterized in that The non-consecutive predetermined time frames are regularly spaced in time.

5. The method of claim 1, further comprising: The error estimates are derived by calculating differences between respective measurements corresponding to the plurality of sets of signals for the selected time frame and respective measurements corresponding to the plurality of sets of signals for the another time frame.

6. The method according to claim 5, characterized in that Calculating the difference includes calculating a difference image frame between the image frames of the plurality of groups of signals with respect to the selected time frame and the image frames of the plurality of groups of signals with respect to the another time frame.

7. The method according to claim 5, characterized in that The error estimate is derived using areas of the calculated differences corresponding to non-illuminated areas of the illumination pattern of the structured light source.

8. The method of claim 6, further comprising: Measurements of the difference image frame corresponding to the non-illuminated area are stored.

9. The method according to any one of claims 1 to 6, characterized in that The first unstructured light source and the structured light source share a substantially common illumination field.

10. The method according to any one of claims 1 to 6, characterized in that The unstructured light source emits a uniform illumination pattern.

11. The method according to claim 7, characterized in that The calculated difference comprises an error measure with respect to the unilluminated area of ​​the illumination pattern of the structured light source; and The derivation of the error estimate includes: Interpolation is performed using the measurements corresponding to the non-illuminated areas of the illumination pattern of the structured light source to obtain error measurements with respect to illuminated areas of the illumination pattern of the structured light source.

12. The method of claim 1, further comprising: The calculated difference measure with respect to the unilluminated area is scaled.

13. The method of any one of claims 1 to 6, further comprising: another structured light source illuminates the scene simultaneously with the illumination of the scene by the unstructured light source, the illumination by the another structured light source occurring during a further predetermined time frame of the plurality of consecutive time frames that is different from the predetermined time frame; and The derivation of the error estimate further comprises employing the plurality of sets of signals generated during another selected one of the further predetermined time frames, illumination of the another unstructured light source occurring during the another selected time frame.

14. An indirect time-of-flight distance calculation device with multipath error compensation, comprising: a structured light source configured to illuminate a scene with an illumination pattern that includes unilluminated areas; a photodetector array comprising a plurality of photodetector elements, the photodetector array being configured to generate a plurality of sets of signals in response to illumination by light reflected from the scene, the plurality of sets of signals including a set of signals substantially simultaneously generated by each of the plurality of photodetector elements in accordance with an indirect time-of-flight measurement technique; processing resources,configured to calculate distances; an unstructured light source configured to illuminate the scene during a plurality of consecutive time frames; and the structured light source being configured to illuminate the scene simultaneously with the illumination of the scene by the unstructured light source, the illumination by the structured light source occurring during a predetermined frame of the plurality of consecutive time frames; the processing resource being configured to derive error estimates from the plurality of sets of signals generated during a selected time frame among the plurality of consecutive time frames and another time frame temporally related to the selected time frame, illumination of the unstructured light source occurring during the selected time frame, and illumination of both the structured light source and the unstructured light source occurring during the another time frame; The processing resource is configured to apply the error estimate to a distance calculation with respect to the scene; and The error estimate is derived with respect to the non-illuminated area using a priori knowledge of the illumination pattern.

Images