Heterodyne depth sensor

Abstract

Heterodyne Depth Sensor This description relates to a heterodyne sensor comprising: one or more optical mixers configured to combine a reference light beam with one or more return light beams to generate one or more beat signals; and a pixel array, each pixel comprising: - a single-photon avalanche diode (SPAD) configured to receive a corresponding beat signal from among the one or more beat signals and to generate an output signal based on the light intensity of the received beat signal; and - a digital counter (CNTR) configured to generate a count value based on the output signal of the single-photon avalanche diode (SPAD). Abstract figure: Fig. 5

Description

Title of the invention: Heterodyne depth sensor Technical field

[0001] The present disclosure relates generally to the field of depth sensors, and in particular to the field of heterodyne depth sensors. Background Art

[0002] One type of heterodyne sensor is a frequency-modulated continuous wave (FMCW) active imaging system, also known as FMCW lidar. In such a system, the light source is a coherent source, for example a laser source, which emits frequency-modulated radiation whose frequency varies according to a periodic linear ramp. The radiation emitted by the source is split into a reference beam and a transmission beam. The transmission beam is projected onto the scene where it is reflected by an object to the sensor's optical mixer, while the reference beam is sent to an optical mixer of the image sensor without passing through the scene.The reference beam and the transmission beam interfere with each other at the optical mixer of the image sensor, resulting in a beat signal having a frequency representative of the time delay between the two beams, which is proportional to twice the distance between the image sensor and the object, thus making it possible to determine the depth of the object in the scene.

[0003] In order to capture a depth image based on FMCW, an array of imaging pixels is for example used to detect the amplitude of the return signal from the scene. However, it is technically difficult to provide an image sensor suitable for FMCM imaging and having acceptable resolution, accuracy, power consumption and area. Summary of the invention

[0004] According to one aspect, a heterodyne sensor is provided comprising: one or more optical mixers configured to combine a reference light beam with one or more return light beams to generate one or more beat signals; and a matrix of pixels, each pixel comprising: - a single-photon avalanche diode configured to receive a corresponding beat signal from one or more beat signals and to generate an output signal based on the light intensity of the received beat signal; and - a digital counter configured to generate a count value based on the output signal of the single-photon avalanche diode.

[0005] According to one embodiment, the matrix comprises pixels arranged in rows and in columns, the heterodyne sensor comprising a sequencer configured to operate the pixels of the matrix during a rolling shutter operation in which the counters of the pixels of each row of the matrix are read during a corresponding read phase which is offset from the read phases of the counters of the pixels of the other rows of the matrix.

[0006] According to one embodiment, the heterodyne sensor further comprises one or more optical elements covering the matrix and configured to direct the return light beam towards the one or more optical mixers.

[0007] According to one embodiment, the sampling frequency of the counter of each pixel is less than 500 kHz, and preferably less than 100 kHz.

[0008] According to one embodiment, the one or more optical mixers are a single element covering the pixel matrix.

[0009] According to another aspect, there is provided a heterodyne imaging system comprising: - the aforementioned heterodyne sensor; - a modulation circuit configured to generate a ramp signal for modulating a coherent light source to generate a modulated light beam; - an optical splitter configured to split the modulated light beam into a reference light beam and a transmission light beam; and - an optical illumination system configured to illuminate a field of view of the heterodyne sensor with the transmission beam.

[0010] According to one embodiment, the heterodyne imaging system further comprises a memory for storing a succession of images generated by the heterodyne sensor, and a post-processing circuit configured to perform a Fourier transform operation on the pixel values ​​captured by each pixel in the succession of images in order to evaluate a frequency variation captured by each pixel, and thus generate a depth value per pixel.

[0011] According to another aspect, there is provided a method of capturing a depth image based on a heterodyne sensor, the method comprising: - the combination, by one or more optical mixers, of a reference light beam and one or more return light beams in order to generate one or more beat signals; - generating, by a single-photon avalanche diode of each pixel of a pixel array of the heterodyne sensor, an output signal as a function of the light intensity of a corresponding beat signal among the one or more beat signals; and - the generation, by a digital counter of each pixel, of a count value based on the output signal of the single-photon avalanche diode.

[0012] According to one embodiment, the matrix comprises pixels arranged in rows and in columns, the method further comprising operating, by a sequencer of the heterodyne sensor, the pixels of the matrix in a rolling shutter operation in which the counters of the pixels of each row of the matrix are read during a corresponding read phase which is offset from the read phases of the counters of the pixels of the other rows of the matrix.

[0013] According to one embodiment, the sampling frequency of the counter of each pixel is less than 500 kHz, and preferably less than 100 kHz.

[0014] According to one embodiment, the method further comprises: - the generation, by a modulation circuit, of a ramp signal for modulating a coherent light source in order to generate a modulated light beam; - the division, by an optical separator, of the modulated light beam into a reference light beam and a transmission light beam; and - illumination, by an optical illumination system, of a field of vision of the heterodyne sensor with the transmission beam.

[0015] According to one embodiment, the method comprises in - the storage in a memory of a succession of images generated by the heterodyne sensor; and - the execution, by a post-processing circuit, of a Fourier transform operation on the pixel values ​​captured by each pixel in the succession of images in order to evaluate a frequency variation captured by each pixel, and thus generate a depth value per pixel. Brief description of the drawings

[0016] These characteristics and advantages, as well as others, will be explained in detail in the following description of particular embodiments given without limitation in relation to the attached figures among which:

[0017] [Fig.l] schematically illustrates an FMCW sensor according to an exemplary embodiment;

[0018] [Fig.2] is a graph showing the frequency of transmitted and received ramp signals as a function of time, according to an exemplary embodiment;

[0019] [Fig. 3] is a graph representing, in the frequency domain, a detected frequency difference between the transmitted and received ramp signals, according to an exemplary embodiment;

[0020] [Fig.4] schematically illustrates an FMCW imaging system according to an exemplary embodiment of the present disclosure;

[0021] [Fig.5] schematically illustrates a heterodyne imager of the FMCW imaging system of [Fig.4] in more detail, according to an exemplary embodiment of the present disclosure;

[0022] [Fig.6] schematically represents a cross-section of the heterodyne imager of [Fig.5] according to an exemplary embodiment of the present description; and

[0023] [Fig.7] is a timing diagram illustrating a rolling shutter operation of the heterodyne imager of [Fig.5] according to an exemplary embodiment of the present disclosure. Description of the embodiments

[0024] The same elements have been designated by the same references in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same references and may have identical structural, dimensional and material properties.

[0025] For the sake of clarity, only the steps and elements useful for understanding the described embodiments have been shown and are detailed. For example, optical mixers for combining two light signals having different modulated frequencies in order to generate a beat signal are known in the state of the art and will not be described in detail in this document.

[0026] Unless otherwise specified, when referring to two elements connected between them, it means directly connected without intermediate elements other than conductors, and when we refer to two elements connected (in English "coupled") between them, it means that these two elements can be connected or be linked by means of one or more other elements.

[0027] In the following description, when reference is made to absolute position qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative position qualifiers, such as the terms "above", "below", "upper", "lower", etc., or to orientation qualifiers, such as the terms "horizontal", "vertical", etc., reference is made unless otherwise specified to the orientation of the figures.

[0028] Unless otherwise specified, the expressions "approximately", "approximately", "sen "possibly", and "of the order of" mean within 10%, preferably within 5%.

[0029] [Fig.l] schematically illustrates an FMCW sensor 100 according to an exemplary embodiment.

[0030] The sensor 100 of [Fig.l] comprises a coherent modulated light source 101, adapted to the emission of a coherent light beam BEAM. The light source 101 comprises for example an FMCW laser modulator (LASER MOD) 102 configured to generate a ramp signal RAMP, for example in the form of a voltage or a current, and to provide the ramp signal to an emission source 104. In certain embodiments, the emission source 104 is implemented by a controlled oscillator configured to modulate the frequency of the beam BEAM in function of the ramp signal RAMP. According to a variant, the emission source 104 is for example a source, such as a VCSEL (from the English "Vertical Cavity Surface-Emitting Laser"), having a natural wavelength which is naturally modulated by a current intensity of the ramp signal, thus creating the frequency modulation without the need for a controlled oscillator. The emission source 104 is for example a laser source, and is for example configured to emit light in the infrared spectrum. The optical frequency of the light source 104 is modulated according to a predefined law, for example a continuous law, for example a periodic linear law. The optical frequency of the source 104 varies over time according to said predefined law.

[0031] The sensor 100 of [Fig.l] further comprises an optical module 106 comprising, for example, an optical splitter 108 adapted to split the light beam BEAM into a transmission beam Tx which is transmitted into the scene, and a reference beam REF, which is supplied to an optical mixer 110. The transmission beam Tx is reflected by one or more target objects (TARGET(S)) in the scene at a given distance(s) from the sensor 100, and the reflected beam forms a return beam Rx towards the optical module 106. The return beam Rx is combined with the reference beam REF, for example by spatially superimposing one beam on the other, in order to generate a heterodyne beam dF, corresponding to a beat signal having a frequency, or a plurality of frequency components, equal to a frequency difference between the transmission and return beams Tx, Rx.

[0032] The sensor 100 further comprises a conversion circuit (FFT) 114, for example comprising a Fourier transform function, such as a fast Fourier transform (FFT) function, configured to sample the heterodyne beam dF to determine one or more frequencies representing one or more distances of target objects in the scene, as will be explained in more detail in relation to FIGS. 2 and 3.

[0033] Figure 2 is a graph showing the frequency variations (FREQ) of the ramps of the transmission beam Tx and the return beam Rx as a function of time (t) according to an exemplary embodiment. Figure 2 is based on an example in which the ramp signal RAMP has a triangular waveform, and has a frequency range B from a minimum frequency Fmin to a maximum frequency Fmax, and an upward sweep duration, i.e. from Fmin to Fmax, of T. The downward sweep duration, i.e. from Fmax to Fmin, is for example equal to the upward sweep duration. The return beam Rx has a similar waveform to that of the transmission beam Tx, but it is delayed by a certain time delay ^tof relative to the transmission beam Tx. This time delay corresponds to the time required for the transmission beam Tx to reach the object target and return to the sensor, and is therefore proportional to twice the distance between the sensor and the target.

[0034] [Fig. 3] is a graph representing, in the frequency domain, the result of a fast Fourier transform (FFT AMPL) applied to the amplitude of the heterodyne beam. As shown, there is a peak at frequency dF corresponding to the frequency difference between the reference beam REF and the return beam Rx reflected by a first target object in the scene. [Fig. 3] also illustrates another peak at a frequency dF' corresponding to the frequency difference between the reference beam REF and another frequency component of the return beam Rx in the case where there is a second target object in the scene at a greater distance than the first target object.

[0035] The distance of an object can be calculated using the following equation:

[0036] [Math.l] d=c-^ OR [Math.l] c' is the speed of light in air. Time [Math.l] hoF can be obtained by the following equation:

[0037] [Math.2] * _ T JF lTOF ~ ~B~ OR [Math.2] T is the time required to perform the frequency sweep, [Math.2] B is the frequency range of the frequency sweep.

[0038] According to one example, the wavelength >1 of the light source varies between 940.000 nm and 940.003 nm, which means that Fmin=319.14554 THz, Fmax=319.14954 THz and B=^ GHz. Furthermore, the duration of the upward scan T is for example equal to 10 ms, and the maximum distance Dmax that can be detected is for example equal to 9.6 m. Therefore, it is possible to define the following parameters: - the minimum distance Dmin that can be detected is equal to c / 2B = 3.75 cm; - FFT granularity for 1 bin=l / T=100 Hz; - the data sampling frequency .T Fs > 25.6 Hz; - the number of points of the FFT output Npoints — 2Dmax.^ — 256.

[0039] More generally, Fmin is for example equal to at least 1 THz, B is equal to at least 100 MHz, Dmax is equal to at least 2 m, and T is equal to at least 1 ms.

[0040] [Fig.4] schematically illustrates an FMCW 400 imaging system according to a exemplary embodiment of the present description. Certain features of the system 400 of [Fig. 4] are the same as those of the sensor 100 of [Fig. 1]. These elements have been designated by similar reference numerals and will not be described again in detail. In particular, the principles of FMCW imaging described in connection with [Fig. 1] are implemented in the system 400.

[0041] The system 400 includes an FMCM laser modulator (FMCW LASER MODULATOR) 102 similar to that of [Fig. 1], configured to generate a ramp signal RAMP, and a coherent light source (COHERENT LIGHT SOURCE) 104 similar to that of [Fig. 1]. In the example of [Fig. 4], the FMCW laser modulator 102 is configured to generate a ramp signal having a sawtooth waveform. In alternative embodiments, instead of a sawtooth waveform, a triangular waveform or another waveform having an ascending and / or descending ramp could be used.

[0042] The beam generated by the light source 104 is provided to the receiving side of the system as a reference beam and is transmitted into the scene as a transmit beam Tx. For example, although not shown in [Fig. 4], the modulated output beam generated by the coherent light source 104 is split using an optical splitter similar to the splitter 108 of [Fig. 1] to generate the reference and transmit beams REF, Tx.

[0043] In the example of [Fig. 4], the system 400 further comprises a flash illumination optical system (FLASH ILLUMINATION OPTICAL SYSTEM) 402 configured to simultaneously illuminate an entire field of view of the imaging system 400 with the transmission beam Tx. For example, the optical system 402 comprises optical components such as diffusers and / or lenses.

[0044] As in [Fig.l], the scene comprises one or more targets (TARGET(S)) 112 at a distance z from the imaging system.

[0045] The return beam Rx coming from the scene is for example received via an optical imaging system (OPT. IMAGING SYSTEM) 404, an optical mixer (MXR) 406 and a heterodyne imager (HETERODYNE IMAGER) 408 of the imaging system 400. The heterodyne imager 408 comprises for example a matrix of pixels Px,y.

[0046] The imaging optical system 404 is for example comprised of one or more lenses configured to direct the return beam toward the optical mixer 406 and the heterodyne imager 408. For example, the imaging optical system 404 includes a single lens covering the field of view of the system. In other embodiments, the one or more lenses could include discrete lenses, one per pixel of the heterodyne imager.

[0047] The beam Rx' generated by the one or more lenses of the imaging optical system 404 is for example provided via the optical mixer MXR to each pixel Pxy of the heterodyne imager 408. The optical mixer MXR is for example a single element that covers the array and is configured to combine the common reference beam REF with the received beam transmitted by the one or more lenses of the imaging optical signal and to provide the resulting beat signal dF to the pixels Px.y of the heterodyne imager 408. In other embodiments, the optical mixer is pixel-based, i.e., an optical mixer element MXR^ is provided for each pixel, each element receiving the reference beam REF and combining it with the received beam to generate the corresponding beat signal dF^;.

[0048] The heterodyne imager 408 is for example configured to sample a light intensity of the beat signal dF received by each pixel, and to provide the sampled signals in the form of digital images of intensity I to a post-processing circuit (POST-PROCESSING) 410. The circuit 410 is for example configured to perform a Fourier transform, such as an FFT operation, on the pixel values ​​captured by each pixel Px,y in the succession of images I in order to evaluate the variation in frequency dF captured by each pixel, and thus generate a depth value (Zv v) per pixel. The circuit 410 is for example implemented in hardware, for example by an ASIC (application specific integrated circuit). According to a variant, the functions of the circuit 410 could be at least partially implemented by software executed by one or more processors of the circuit 410.

[0049] Figure 5 schematically illustrates the heterodyne imager 408 of the FMCW imaging system 400 of Figure 4 in more detail according to an exemplary embodiment of the present disclosure. The imager 408 comprises a pixel array 502 of size X by Y, in other words, there are pixels -Pij to Pxa in a first row, and pixels Pix to Px,y in a last row.

[0050] Each pixel Px,y of the matrix 502 comprises for example a single photon avalanche diode detector (SPAD) and a counter (CNTR). As known to those skilled in the art, a SPAD detector is configured to generate an event, such as a voltage transition or a pulse, each time the arrival of a photon is detected. According to the embodiments described here, the events are accumulated locally by the CNTR counter in each pixel. Each CNTR counter is for example reset by a reset signal RESET, and read by a read signal READ. For example, the reset and read signals applied to each counter have a frequency equal to the sampling frequency Fs of the imager. The sampling frequency Fs is for example equal to twice the maximum frequency of the signal dF to be detected. According to one embodiment, for a maximum detection frequency of 25.6 kHz, the sampling frequency Fs is 51.2 kHz.

[0051] The example imager 408 of [Fig. 5] is based on a rolling shutter operation in which the rows are individually controlled by read and reset signals generated by a sequencer (SEQ.) 504. For example, assuming that the matrix has n rows from 0 to (n-1), the CNTR counters of a first row 0 of the matrix are each controlled by a common read signal READ[0] and a common reset signal RESET[0], and the CNTR counters of a last row of the matrix are each controlled by a common read signal READ[nl] and a common reset signal RESET[nl].

[0052] The CNTR counters of each column of pixels are for example configured to provide output data in a common column data bus COLUMN DATA, this column data forming the digital images I which are for example stored in a memory (not shown in [Fig.5]) and processed by the post-processing circuit 410.

[0053] Figure 6 schematically represents a cross-section of a portion of the heterodyne imager of Figure 5 according to an exemplary embodiment of the present disclosure. The cross-section of Figure 6 illustrates in particular the pixels a and P2,1 of [Fig.5], but it is understood that each of the pixels of the array 502, for example, has a similar structure. As shown, each pixel comprises for example a stack formed in and on a substrate 602, the stack comprising a SPAD detector (SPAD), an optical mixer (MXR), which may be a common element for the entire array or a pixel-based element, covering the SPAD detector, a separation layer 604, for example made of a transparent material such as oxide (not drawn to scale), and an optical element (LENS) formed on the separation layer 604, the optical element being for example a single lens covering the entire array.

[0054] [Fig.7] is a timing diagram illustrating a rolling shutter operation of the heterodyne imager of [Fig.5] according to an exemplary embodiment of the present disclosure.

[0055] As shown, the example of Figure 7 is based on an operation of rolling shutter in which the n rows ROW(0) to ROW(nl) of the matrix are individually controlled. The digital counter CNTR of each pixel of a row is reset by the corresponding reset signal RESET before an integration period (INT) of the row, and is read by the corresponding read signal READ at the end of the integration period (INT) of the row. A sampling period (SAMPLE PERIOD) extends from the end of the reset phase of a counter to the beginning of a subsequent reset phase of the counter. During a sampling period, the n rows of the matrix are, for example, read. Thus, the sampling periods between one row and the next are, for example, shifted by a time delay equal to the sampling period divided by n.Thus, the sequencer 504 is configured to operate the pixels of the matrix in a rolling shutter operation whereby the CNTR counters of the pixels of each row of the matrix are read during a corresponding read phase which is offset from the read phases of the counters of the pixels of the other rows of the matrix.

[0056] A sequence of m sampling periods is for example executed in order to determine, for each pixel, one or more depth readings. In certain embodiments, m is equal to at least 10, and preferably to at least 100.

[0057] An advantage of the embodiments described herein is that depth images having a relatively high resolution can be captured by a heterodyne imager operating at a relatively low sampling rate, e.g., less than 500 kHz, and preferably less than 100 kHz. For example, the pixel array can comprise 64 or more pixels, arranged, for example, in 8 or more rows and 8 or more columns, and preferably at least 500 pixels arranged in two or more rows and two or more columns.

[0058] Furthermore, an advantage of the rolling shutter operation described in connection with [Fig.7] is that the rise time T of the ramp signal can be substantially equal to the full period of the image, thereby achieving a high signal-to-noise ratio (SNR) and relaxing constraints on the sampling frequency.

[0059] The use of a SPAD detector as a photodetector in each pixel of the heterodyne imager has the advantage of having a relatively fast response time and of being compatible with the control of a local digital counter in order to locally store an intensity measurement in each pixel. Thus, SPAD-based detection allows digital conversion to be performed directly in the pixel with relatively low power and surface area.

[0060] Various embodiments and variations have been described. Those skilled in the art will understand that certain features of these various embodiments and variations could be combined, and other variations will occur to those skilled in the art. profession. For example, although the described embodiments provide that each pixel of an array comprises a single SPAD, it would be possible for each pixel to comprise a plurality of SPADs.

[0061] Finally, the practical implementation of the embodiments and variants described is within the reach of those skilled in the art from the functional indications given above.

Claims

Claims

1. A heterodyne sensor comprising: - one or more optical mixers (MXR) configured to combine a reference light beam (REF) with one or more return light beams (Rx ') to generate one or more beat signals (dF); and a pixel array (Px^, each pixel comprising: - a single photon avalanche diode (SPAD) configured to receive a corresponding one of one or more beat signals (dF) and to generate an output signal based on the light intensity of the received beat signal (dF); and - a digital counter (CNTR) configured to generate a count value based on the output signal of the single photon avalanche diode (SPAD).

2. The heterodyne sensor of claim 1, wherein the matrix comprises pixels (Ay ) arranged in rows and columns, the heterodyne sensor comprising a sequencer (504) configured to operate the pixels of the matrix in a rolling shutter operation in which the counters (CNTR) of the pixels of each row of the matrix are read during a corresponding read phase which is offset relative to the read phases of the counters of the pixels of the other rows of the matrix.

3. A heterodyne sensor according to claim 1 or 2, further comprising one or more optical elements (LENS) covering the matrix and configured to direct the return light beam (Rxx.y) towards the one or more optical mixers (MXR).

4. Heterodyne sensor according to one of claims 1 to 3, in which the sampling frequency (Fs) of the counter of each pixel is less than 500 kHz, and preferably less than 100 kHz.

5. A heterodyne sensor according to one of claims 1 to 4, wherein the one or more optical mixers are a single element (MXR) covering the pixel matrix.

6. A heterodyne imaging system comprising: - the heterodyne sensor according to one of claims 1 to 5; - a modulation circuit (102) configured to generate a ramp signal (RAMP) for modulating a coherent light source (104) in order to generate a modulated light beam; - an optical splitter (108) configured to split the modulated light beam into a reference light beam (REF) and a transmission light beam (Tx); and - an illumination optical system (402) configured to illuminate a field of view of the heterodyne sensor with the transmission beam (Tx).

7. The heterodyne imaging system of claim 6, further comprising a memory for storing a succession of images (I) generated by the heterodyne sensor, and a post-processing circuit (410) configured to perform a Fourier transform operation on the pixel values captured by each pixel (A,.y) in the succession of images (I) to evaluate a frequency variation (dF) captured by each pixel, and thereby generate a depth value (Zxv) per pixel.

8. A method for capturing a depth image based on a heterodyne sensor, the method comprising: - combining, by one or more optical mixers (MXR), a reference light beam (REF) and one or more return light beams (Æx) to generate one or more beat signals (dF); - generating, by a single-photon avalanche diode (SPAD) of each pixel (^..y) of a pixel array of the heterodyne sensor, an output signal as a function of the light intensity of a corresponding one of the one or more beat signals (dF); and - generating, by a digital counter (CNTR) of each pixel (A,y), a count value based on the output signal of the single-photon avalanche diode.

9. The method of claim 8, wherein the matrix comprises pixels arranged in rows and columns, the method further comprising operating, by a sequencer (504) of the heterodyne sensor, the pixels (A;y) of the matrix in a rolling shutter operation in which the counters (CNTR) of the pixels of each row of the matrix are read during a corresponding read phase which is offset relative to the read phases of the counters of the pixels of the other rows of the matrix.

10. Method according to claim 8 or 9, wherein the sampling frequency (Fs) of the counter of each pixel is less than 500 kHz, and preferably less than 100 kHz.

11. A method according to any one of claims 8 to 10, further comprising: - generating, by a modulation circuit (102), a ramp signal (RAMP) for modulating a coherent light source (104) to generate a modulated light beam; - splitting, by an optical splitter (108), the modulated light beam into a reference light beam (REF) and a transmission light beam (Tx); and - illuminating, by an illumination optical system (402), a field of view of the heterodyne sensor with the transmission beam (Tx).

12. The method of claim 11, further comprising: - storing in a memory a succession of images (I) generated by the heterodyne sensor; and - performing, by a post-processing circuit (410), a Fourier transform operation on the pixel values captured by each pixel (Px.y) in the succession of images (I) in order to evaluate a frequency variation (dF) captured by each pixel, and thus generate a depth value per pixel.