Light time-of-flight sensor

The time-of-flight camera system addresses power loss by arranging pixels in a line with adjacent modulation and integration nodes, enabling grouped switching and partial deactivation of pixel columns to optimize power efficiency.

DE102021112402B4Active Publication Date: 2024-12-05IFM ELECTRONIC GMBH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
DE102021112402
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-05-12
Publication Date
2024-12-05
Estimated Expiration
2041-05-12

AI Technical Summary

Technical Problem

Existing time-of-flight sensors face challenges in reducing power loss and optimizing power consumption, particularly in applications requiring high spatial resolution and low temporal resolution.

Method used

The time-of-flight camera system employs a design with time-of-flight pixels arranged in a line, where modulation gates and integration nodes are adjacent, allowing for grouped switching via modulation drivers, and partial areas of the pixel columns are deactivated to reduce power consumption.

Benefits of technology

This approach effectively reduces power consumption by switching off entire pixel columns, maintaining high spatial resolution while minimizing power usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Light time-of-flight camera with a time-of-flight sensor (22) with several time-of-flight pixels (21) arranged in a row, wherein the light-time-of-flight pixels (21) have modulation gates (A, Gam, B, Gbm) and integration nodes (Ga, Gb) arranged such that the modulation gates (A, Gam, B, Gbm) and the integration nodes (Ga, Gb) of adjacent light-time-of-flight pixels (21) are arranged next to each other and the modulation gates (A, Gam, B, Gbm) and the integration nodes (Ga, Gb) each form their own row, with a modulator for generating a modulation signal to operate the modulation gates (A, Gam, B, Gbm), which is connected to multiple modulation drivers (S), wherein the modulation drivers (S) have at least one A-channel and at least one B-channel signal line and each channel signal line (A, B) is connected to two modulation gates (Gam, Gbm) belonging to the channel, wherein the time-of-flight camera is configured such that that when the pixel row is illuminated, the modulation drivers (S) are activated, whose assigned pixels or modulation gates (Gam, Gbm) are illuminated by a light spot. and secondly, one or two adjacent modulation drivers (S) are added if pixels or modulation gates (Gam, Gbm) adjacent to the light spot are not fully modulated by the actively switched modulation drivers, with all other modulation drivers (S) not being operated.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The invention relates to a time-of-flight sensor according to the preamble of the independent claim.

[0002] The time-of-flight sensor particularly relates to time-of-flight camera systems, particularly time-of-flight or 3D-TOF camera systems that obtain time-of-flight information from the phase shift of emitted and received radiation. PMD cameras with photonic mixer detectors (PMDs), as described, among others, in applications EP 1 777 747 B1, US 6 587 186 B2, and also DE 197 04 496 C2, are particularly suitable as time-of-flight or 3D-TOF cameras. The PMD camera, in particular, allows for a flexible arrangement of the light source and detector, which can be arranged either in one housing or separately.

[0003] Furthermore, DE 198 21 974 A1 discloses a device for detecting the phase and amplitude of electromagnetic waves, in which light-transmitting modulation gates are arranged in the photosensitive area and charge-collecting accumulation gates are arranged in strip form.

[0004] Furthermore, WO 2019 / 050606 A1 discloses a TOF sensor for measuring distances. It comprises a matrix of TOF pixels with modulation and integration gates. The TOF sensor also includes a multiplexer that can be used to select one or more pixels for further signal processing.

[0005] The object of the invention is to reduce the power loss.

[0006] The object is advantageously achieved by the light transit time sensor according to the invention according to the preamble of the independent claim.

[0007] Advantageously, a time-of-flight camera is provided, with a time-of-flight sensor with several time-of-flight pixels arranged in a row, wherein the time-of-flight pixels have modulation gates and integration nodes arranged such that the modulation gates and the integration nodes of adjacent time-of-flight pixels are arranged next to each other and the modulation gates and the integration nodes each form a separate row, with a modulator for generating a modulation signal for operating the modulation gates, which is connected to several modulation drivers, wherein the modulation drivers have at least one A- and at least one B-channel signal line, and each channel signal line is connected to two modulation gates associated with the channel.

[0008] This approach has the advantage that pixels can be switched on or off in groups via the modulation drivers.

[0009] The invention is explained in more detail below using exemplary embodiments with reference to the drawings.

[0010] They show schematically: Fig. 1 the basic principle of a time-of-flight camera based on the PMD principle, Fig. 2 a modulated integration of the time-shifted generated charge carriers, Fig. 3 a cross-section of a PMD pixel, Fig. 4 a top view of a PMD pixel, Fig. 5 a top view of a PMD pixel with readout nodes, Fig. 6 a top view of a PMD pixel with forked readout fingers, Fig. 7 a pixel line with vertically arranged pixels, Fig. 8 a driver concept for a pixel line according to Fig. 8, Fig. 9 an operating mode of an illuminated pixel line, Fig. 10 controlling the pixel line when the lighting changes, Fig. 11 a pixel line with several vertically arranged pixels, Fig. 12 a possible driver control, Fig. 13 a basic structure of a triangulation measurement.

[0011] In the following description of the preferred embodiments, like reference numerals designate like or comparable components.

[0012] Fig. 1 shows a measuring situation for an optical distance measurement with a time-of-flight camera, as known, for example, from DE 197 04 496 C2.

[0013] The time-of-flight camera system 1 comprises a transmitting unit or illumination module 10 with an illumination light source 12 and associated beam-shaping optics 15, as well as a receiving unit or time-of-flight camera 20 with receiving optics 25 and a time-of-flight sensor 22. The time-of-flight sensor 22 has at least one time-of-flight pixel 21, but preferably a pixel array, and is designed in particular as a PMD sensor. The receiving optics 25 typically consist of several optical elements to improve the imaging properties. The beam-shaping optics 15 of the transmitting unit 10 is preferably designed as a reflector. However, diffractive elements or combinations of reflective and diffractive elements can also be used.

[0014] The measuring principle of this arrangement is essentially based on the fact that the time of flight of the emitted and reflected light can be determined based on the phase shift of the emitted and received light. For this purpose, the light source 12 and the time-of-flight sensor 22 are jointly subjected to a specific modulation frequency or modulation signal with a first phase position a via a modulator 30. Depending on the modulation frequency, the light source 12 emits an amplitude-modulated signal with phase a. In the illustrated case, this signal or the electromagnetic radiation is reflected by an object 40 and, due to the distance traveled, impinges on the time-of-flight sensor 22 with a second phase position b correspondingly phase-shifted.In the time-of-flight sensor 22, the signal of the first phase position a of the modulator 30 is mixed with the received signal, which has the time-of-flight-dependent second phase position b, wherein the phase shift or the object distance d is determined from the resulting signal.

[0015] For a more precise determination of the second phase position b and thus the object distance d, it may be provided to change the phase position a with which the time-of-flight sensor 22 is operated in order to adjust predefined phase shifts Δφ. Similarly, it may also be provided to specifically shift the phase with which the illumination is driven.

[0016] The principle of phase measurement is shown schematically in Fig. 2. The upper curve shows the temporal progression of the modulation signal used to control the illumination 12 and the time-of-flight sensor 22, here without phase shift. The light b reflected from the object 40 hits the object according to its time of flight t. L phase-shifted to the time-of-flight sensor 22. The time-of-flight sensor 22 collects the photonically generated charges q during the first half of the modulation period in a first readout finger Ga and in the second half of the period in a second readout finger Gb. The charges are typically collected or integrated over several modulation periods. The phase shift and thus the distance of the object can be determined from the ratio of the charges qa, qb collected in the first and second gates Ga, Gb.

[0017] As already known from DE 197 04 496 C2, the phase shift of the light reflected from the object, and thus the distance, can be determined, for example, using a so-called IQ (in-phase quadrature) method. To determine the distance, two measurements are preferably performed with modulation signals whose phase positions are shifted by 90°. For example, φ mod + φ0 and φ mod + φ 90 , whereby the phase shift of the reflected light can be determined from the charge difference Δq(0°), Δq(90°) determined in these phase positions using the well-known arctan or arctan2 relationship. φ=arctanΔq(90°)Δq(0°)

[0018] To improve accuracy, further measurements can be carried out with phase positions shifted by, for example, 180°. φ=arctanΔq(90°)−Δq(270°)Δq(0°)−Δq(180°) Of course, measurements with more than four phases and their multiples and a correspondingly adapted evaluation are also conceivable.

[0019] Fig. Figure 3 shows a cross-section through a pixel of a photonic mixer detector as known, for example, from DE 197 04 496 C2. The modulation photogates G am , G0, G bm form the light-sensitive area of ​​a PMD pixel. According to the modulation gates G am , G0, G bm applied voltage, the photonically generated charges q are transferred either to one or the other readout finger G a , G b directed. In the example shown, the readout finger is designed as a pn junction or diode.

[0020] Fig. Figure 3b shows a potential curve where the charges q are directed towards the first readout finger G a while the potential according to Fig. 3c the charge q in the direction of the second readout finger Gb The potentials are specified according to the applied modulation signals. Depending on the application, the modulation frequencies are preferably in the range of 1 to 100 MHz. For example, a modulation frequency of 1 MHz results in a period of one microsecond, so the modulation potential changes every 500 nanoseconds.

[0021] In Fig. 3a also shows a readout unit 400, which may already be part of a PMD time-of-flight sensor designed as a CMOS. The readout finger G a , G bThe detected photonically generated charges are collected over a plurality of modulation periods. In a known manner, the voltage then present at the readout fingers Ga, Gb can be tapped at a high impedance, for example, via the readout unit 400. The integration times should preferably be selected such that the time-of-flight sensor or the readout fingers, or their capacitances and / or the light-sensitive areas, do not reach saturation for the expected amount of light.

[0022] Fig. Figure 4 shows a top view of a time-of-flight pixel 23 with typical length specifications. The distance or length between the two readout fingers Ga, Gb is referred to as the channel spacing L K and the distance between the longitudinal symmetry axes of the two readout fingers Ga, Gb as finger pitch L FPAccording to the invention, the pixels are not separated from one another by a field oxide, so that a further modulation gate is directly connected to the readout finger.

[0023] The extension of the gates or readout fingers parallel to the channel spacing L K is the gate length L G and the perpendicular extension is the gate width B G While the gates in their latitudes B G are essentially the same length, the gate lengths are typically varied depending on the application. In the illustrated case, for example, the middle modulation gate G0 is longer than the two outer modulation gates G am , G bm , for example to influence a modulation contrast.

[0024] The Auslesefingers G a , G bcan be constructed and structured in different ways. For example, it is known from DE 197 04 496 C2 to design the readout fingers as pn junctions or diodes. The diode preferably extends across the entire width of the readout fingers. The readout fingers are preferably opaque or covered with an opaque layer, preferably a metallization. The metallization can also simultaneously form the contact for the diode.

[0025] Fig. Figure 5 shows an embodiment in which the readout fingers Ga, Gb have a gate structure and an integration or readout node. In the illustrated case, a readout gate AG extends a , AG b across the entire width B G of the readout finger Ga, Gb. In one end region of the readout finger Ga, Gb there is a readout node AK a , AK b As already mentioned in Fig. 3 shown, is located at the Auslesegate AG a , AG b a voltage that forms a potential well for the charge carriers q below the gate. The charge carriers collected there flow to the respective readout node AK primarily due to diffusion processes. a , AK b and can be tapped there by the readout unit 400, for example as voltage.

[0026] Preferably, the readout fingers Ga, Gb are covered with an opaque layer, preferably a metal layer. The metal layer can also serve to contact the readout node. In a further embodiment, it is also conceivable to influence the transparency of the readout gate itself, for example, by silicide coating.

[0027] Furthermore, variations in the size and position of the readout nodes AK a , AK b For example, the readout nodes AK a , AK bthe same length as the Auslesegates AG a , AG b Multiple readout nodes can also be distributed across the gate structure. In particular, readout nodes can be arranged at both ends of the gate. A central readout node is also conceivable. The readout nodes are preferably designed as a pn junction or a diode.

[0028] Fig. Figure 6 shows another variant in which the readout fingers or gate structure are extended in the area of ​​the readout nodes. This branching allows the readout nodes AK a , AK b surrounded by a gate structure and separated by a minimum distance from the neighboring modulation gate Gam, Gbm. This approach advantageously reduces or inhibits electrical penetrations from the modulation gate to the readout nodes.

[0029] As already mentioned, a PMD pixel is typically separated from the neighboring matrix pixel using a field oxide and an optical cover. This typically minimizes crosstalk between the individual matrix pixels. However, for very small pixels, the fill factor plays an increasingly important role. It is particularly disadvantageous if the separating field oxide matrix has a larger dimension than the readout fingers. If the pixel pitch is on the same order of magnitude as the finger pitch, the edge region or separation region plays an increasingly important role with respect to the fill factor. The typical finger pitch L FP , i.e., readout fingers plus modulation gates, is on the order of approximately 7 - 10 µm. The height and width of the fingers and the channel length of PMD structures are scalable over a very wide range, similar to transistors.

[0030] Typically, a PMD sensor captures four frames sequentially with four different phase angles. While this allows for high spatial resolution, it results in low temporal resolution.

[0031] In a pixel row or matrix, often only a specific sub-area is required, which is then actually read out. Instead of modulating the entire pixel row, the invention provides for only a sub-area to be modulated. For example, four columns of a modulation driver can be deactivated individually. The deactivated areas receive no clock and therefore consume virtually no power. The goal is to reduce power consumption. According to the invention, entire pixel columns are deactivated for this purpose.

[0032] As an alternative to the ToF principle, as in Fig. 13, as shown by way of example, can also be measured using triangulation. In this case, the distance is determined from the angle of the incident light. The transmitter (VCSEL or laser diode) and receiver are not in exactly the same position. The distance between them is called the eye distance. The receiver therefore sees the reflected light from a specific angle. This can then be used to determine the distance. If the receiver consists of a pixel row consisting of many pixels, the light spot hits a specific position within the pixel row. The position is calculated from the pixel signals, e.g. using a center of gravity, and from this the distance to the object is calculated.

[0033] Triangulation works best for short distances ("close range"). For long distances ("far range"), measurement using the ToF principle is recommended. However, if you want to measure larger distances using triangulation, very small pixels must be used to increase accuracy. Furthermore, each pixel column must be read and evaluated individually.

[0034] Fig. Figure 7 shows a pixel row structure according to the invention, particularly optimized for triangulation. The time-of-flight pixels are arranged in a row, with the modulation gates Gam, Gbm and the integration nodes Ga, Gb of the time-of-flight pixels aligned perpendicular to the direction of the pixel row, so that the modulation gates Gam, Gbm and the integration nodes Ga, Gb each form a separate row. According to the invention, a modulation driver S0, S1, ... each controls two pairs of A- and B-channel modulation gates Gam, Gbm of adjacent pixels.

[0035] The modulation driver S1 controls the A-channel modulation gates Gam of pixels 4, 5, 6 and 7 with its two A-channel signal lines and the B-channel modulation gates Gbm of pixels 3, 4, 5 and 6 with its two B-channel signal lines.

[0036] According to the invention, it is provided that not individual channel signal lines A, B are switched on or off, but always only one modulation driver S as a whole. If all four pixels of the pixels assigned to a modulation unit S are to be operated, it is necessary that the adjacent modulation unit is also operated. In the example shown, pixels 4 to 7 are assigned to the modulation driver S1. The modulation driver S1 completely supplies the modulation gates Gam, Gbm of pixels 4, 5 and 6. In pixel 7, however, only the A-channel modulation gate Gam is supplied by the modulation driver S1. To fully operate this pixel, it is therefore necessary that the neighboring modulation driver S2 is also switched on, which then applies a B-channel signal to the B modulation gate Gbm of pixel 7.

[0037] As in Fig. As further shown in Figure 7, the pixel row is delimited at the edge by an edge pixel.

[0038] Furthermore, in the example according to Fig. 7 shows an edge pixel R that delimits the first modulation driver S0 at the right edge. The first pixel 0 of the pixel row is fully supplied with modulation signals by the first modulation driver S0 through an A- and B-channel signal line. For reasons of symmetry, the B-channel signal line is connected not only to the modulation gate Gbm of the first pixel 0 but also to a 'pseudo-modulation gate' of the edge pixel R. This ensures that the first pixel 0, and in particular the B-channel modulation gate Gbm of the first pixel 0, also has another 'gate partner' for reasons of symmetry. The edge pixel R does not serve as a functioning pixel, but merely as the electrical termination of the pixel row.

[0039] Furthermore, for symmetry reasons, it is planned to set the uncontrolled modulation gate, here Gam, and possibly also the unused integration nodes Ga and Gb at the edge pixel R to a fixed potential Vfix. This approach advantageously prevents unfavorable potential shifts in the edge region. If necessary, the integration nodes could also be omitted entirely from the layout.

[0040] Fig. Figure 8 shows a control structure according to the invention in which, as previously shown, the modulation gates Gam, Gbm or vmoda, vmodb of two adjacent pixels are combined into a section, so that two A-channel modulation gates vmoda and two B-channel modulation gates vmodb can always be switched on or off by controlling the section via en_single_mdrv. In the example shown, the pixel row has pixels from 0 to 183 or modulation gate pairs from 0 to 91, which are each combined in pairs into sections 0 to 45 or modulation drivers S0 to S45.

[0041] All information about which of the 46 modulation drivers are on or off is transmitted serially from the digital part to the digital_mdrv_control control block via a single data_in line. The digital_mdrv_control control block performs a serial-to-parallel conversion of the data. The digital_mdrv_control control block and the digital part are located far apart in the layout. Therefore, it is advantageous that only a single line for the data and a clock line is required, rather than 46 lines. The lines from the digital_mdrv_control control block to the individual modulation drivers S0 to S45 are preferably the same length and can be configured as a cable tree if necessary.

[0042] Fig. Figure 9 shows an illuminated pixel row. For clarity, the existing modulation gates have been omitted. In the illustrated case, a light spot is intended to illuminate pixels 8 to 13. To evaluate the position on the pixel row and to determine the distance according to the phase measurement principle, only the illuminated pixels 8 to 13 are read. The remaining pixels 0 to 7 and 14 to 182 are not read and could, in principle, also be switched off.

[0043] According to the invention, however, it is provided that the pixels adjacent to the illuminated pixel are also operated in a modulated manner but not read out. In the present case, these would be pixels 7 and 14. To operate pixels 8 to 13, modulation drivers S2 and S3 are already switched on, so that pixels 8 to 14 already receive a modulation signal. Pixel 14, which must be operated for reasons of symmetry, is thus already supplied by modulation driver S3. However, pixel 8, which must also be operated for reasons of symmetry, is not fully supplied by modulation driver S2, so that modulation driver S1 is also switched on for complete control. All other modulation drivers remain switched off.

[0044] In Fig. At 10, the spot has moved further to the left and now illuminates pixels 9 to 14, so for symmetry reasons, pixels 8 and 15 now need to be modulated. Pixel 8 is already supplied by the third modulation driver S2, while the fourth modulation driver S4 needs to be activated for pixel 15. The previously activated second modulation driver S1 is then no longer required and, like all other modulation drivers, is deactivated.

[0045] Fig. Figure 11 shows a further embodiment in which a modulation driver is assigned not just four pixels, but four pixel columns of four pixels each. In the example shown, four pixels per column are shown. However, other pixel numbers per column are also conceivable, in particular 2 or 3 pixels.

[0046] Fig. Figure 12 shows a possible circuit for controlling the modulation drivers. Using the signal bus en_single_mdrv, here with an example bus width of 46, four columns of the modulation driver can be individually deactivated. The sens_mdrv_mux block is used to implement the modulation fader. In this example, this exists 46 times, with one block for each of the four columns. If a single en_single_mdrv signal is set to 0, the corresponding four columns behave as if they were in V-mode, and the clock is deactivated for these columns. The sens_mdrv_mux cell consists of an inverter and a NAND gate. If en_single_mdrv is set to 0 in this cell, the output is also set to 0. If en_single_mdrv is set to 1, the output is considered to be the input.

[0047] If only some of the signals on the en_single_mdrv bus are set to 1, only part of the modulation driver is active, while the rest is in V-mode. A different number of columns than four can be used, or each column can be deactivated individually. A separate line is required for the corresponding en_single_mdrv control signal for each four columns.

[0048] To avoid all these lines having to run to the digital part, they are, as shown in Fig.8, connected to a separate control block, digital_mdrv_control, near the modulation driver. This control block encodes the information about which columns should be deactivated, so that only one line (data_in) is required to reach the digital section. The information from the mod_col<11:0> register is transmitted serially on this line. The 6 LSBs determine the bit of en_single_mdrv<45:0> where the modulated range begins. The 6 MSBs determine the bit where the modulated range ends. Example: mod_col<11:0> = 0b101100 000010, then en_single_mdrv<45:> = 0111...11100. The control block requires a clock input. The clock for this control block is only applied when it is actually needed. List of reference symbols 10 transmitter unit 12 Illumination light source 15 Beam shaping optics 20 Receiver unit, TOF camera 22 Light transit time sensor 23 time-of-flight pixels 23Z time-of-flight pixel line 25 Receiving optics 26 light-sensitive area 26a common light-sensitive area 27 light-insensitive area 28a active area for readout finger Ga 28b active area for read finger Gb 30 Modulator 35 Potential supply line 40 objects 400 reading unit 420 evaluation unit Gam, G0, Gbm, MOD modulation gates Ga, Gb selection finger AG a , AG b Readout gates AK a , AK b Readout node q charges qa, qb Charges at the readout finger Ga, Gb

Claims

[1] Time-of-flight camera, with a time-of-flight sensor (22) with several time-of-flight pixels (21) arranged in a row, wherein the time-of-flight pixels (21) have modulation gates (A, Gam, B, Gbm) and integration nodes (Ga, Gb) arranged such that the modulation gates (A, Gam, B, Gbm) and the integration nodes (Ga, Gb) of adjacent time-of-flight pixels (21) are arranged next to one another and the modulation gates (A, Gam, B, Gbm) and the integration nodes (Ga, Gb) each form a separate row, with a modulator for generating a modulation signal for operating the modulation gates (A, Gam, B, Gbm), which is connected to several modulation drivers (S), wherein the modulation drivers (S) have at least one A- and at least one B-channel signal line and each channel signal line (A, B) is connected to two modulation gates (Gam, Gbm) associated with the channel, wherein the time-of-flight camera is designed such that that when the pixel row is illuminated, the modulation drivers (S) are activated, the associated pixels or modulation gates (Gam, Gbm) of which are illuminated by a light spot and on the other hand, one or two adjacent modulation drivers (S) are switched on if pixels or modulation gates (Gam, Gbm) adjacent to the light spot are not fully modulated by the actively switched modulation drivers, whereby all other modulation drivers (S) are not operated.

Citation Information

Patent Citations

  • Method and device for determining the phase and / or amplitude information of an electromagnetic wave

    DE19704496C2

  • Device and method for the demodulation of modulated electromagnetic wave fields

    EP1777747B1

  • CMOS-compatible three-dimensional image sensing using reduced peak energy

    US6587186B2

  • Phase and amplitude detector of electromagnetic waves from pixels operating in visible, UV and IR wavelengths

    DE19821974A1

  • Time-of-flight sensor readout circuit

    WO2019050606A1