Imaging Sensors

Abstract

Imaging 3D time-of-flight sensors are particularly suitable for detecting objects and people in three-dimensional space. This capability comes from the basic principle. At each image point of the sensor, the distance to the object located in the observation space is determined based on the propagation time (time of flight) of the light pulse. The sensor thus provides a three-dimensional image that can be analyzed using corresponding processing algorithms. In certain application cases, especially in the case of interaction between man and machine, machine and machine or machine and space, the detection must be carried out safely. The safety level is classified as safety integrity level (SIL) according to different standards.

Description

[0001] summary

[0002] Imaging 3D time-of-flight sensors (TOF sensors) are particularly suitable for detecting objects and persons in three-dimensional space. This capability is derived from the basic principle. In each image point of the sensor, the distance to an object located in the acquisition area of ​​the sensor is determined based on the propagation time of the light pulse, also called the time of flight (tof). The sensor thus provides a sensor output signal, which reflects a data set in which a distance value is contained for each image point (also called pixel) of the sensor, which corresponds to the distance between the respective image point and the surface segment of the object mapped onto the image point. The sensor thus provides a three-dimensional image, which can be analyzed using corresponding processing algorithms. For example, it is known to analyze the sensor output signal as follows: whether the object is located within a predetermined spatial sector. In specific applications, one or more imaging 3D time-of-flight sensors can be used to detect whether people and machine parts are dangerously close, for example in the machine action area.

[0003] In certain application cases, in particular in the case of interaction between man and machine, machine and machine or machine and room, the detection must be carried out particularly reliably, ie safely.

[0004] Safety levels are classified into the following safety integrity levels (SIL) according to different standards:

[0005] Automotive (ISO 26262) QM ASIL-A ASIL-B / C ASIL-D -

[0006] General (IEC-61508) - SIL-1 SIL-2 SIL-3 SIL-4

[0007] Aviation (DO-178 / 254) DAL-E DAL-D DAL-C DAL-B DAL-A

[0008] Railway (CENELEC - SIL-1 SIL-2 SIL-3 SIL-4 50126 / 128 / 129) Background Art

[0009] 3D-TOF sensors are known, which are arranged with at least one optical transmitter for emitting electromagnetic radiation preferably (but not exclusively) in the near infrared range and an image recorder, which is constructed according to an arrangement of radiation receivers. Each radiation receiver corresponds to a pixel (image point) of the image recorder and thus to a pixel of the 3D-TOF sensor.

[0010] Typically, the radiation receivers are divided into two groups (radiation receiver A, radiation receiver B) and arranged in a matrix.

[0011] Each radiation receiver has a photosensitive element (eg, a photodiode or a photogate) and at least one storage capacitor.

[0012] The typical working principle of a TOF sensor is as follows:

[0013] At time t0, the transmitter is activated, which emits a light pulse for a short time period ti (for example 30ns), which has a light pulse duration corresponding to ti. At the same time, the radiation receiver A is also activated for a short activation time period tA (for example also 30ns). The light pulse is reflected on the object and recorded by the radiation receiver A with a flight time offset tof. During the activation time period tA, the energy of the reflected light pulse causes the charge (i.e., photoelectrons) in the photosensitive element to be released. The amount of charge formed by the recording of the light pulse (corresponding to the number of released photoelectrons) is stored on the storage capacitor (A1). Receiver A can only record the part of the reflected light pulse that corresponds to the time difference tA-tof, because the reflected light pulse hits the photosensitive element with a delay of the flight time tof relative to the start of the activation time period, so that the activation time period and the pulse duration of the reflected light pulse only partially overlap in time for the overlap duration. The overlap duration is inversely proportional to the propagation time or flight time of the light pulse (see Figure 1 ). Accordingly, the number of released photoelectrons is also at least approximately inversely proportional to the propagation time or flight time of the light pulse.

[0014] With a time delay Δ=t0+tA (again 30 ns in this example), i.e. directly following the end of the first activation period tA, the radiation receiver B is activated. The radiation receiver B is active for a second activation period tB (also 30 ns in this example). Likewise, the receiver records only the part of the reflected light pulse which corresponds to the time difference Δ=tof+ti-tA (tof if ti=tA) and is therefore proportional to this time difference (propagation time or flight time of the light pulse). The corresponding charge quantity is stored on the storage capacitor B1.

[0015] Since the reflected signal may be very weak depending on the reflectivity and distance of the reflecting object, an amplification mechanism must be provided, which is achieved by repeating the above-described measurement multiple times and accumulating the corresponding recorded charge on the storage capacitor until a sufficient signal level is reached.

[0016] So the distance can be calculated as follows:

[0017] E1=P(R)*kref*(T-tof)

[0018] E2=P(R)*kref*(tof+TT)=P(R)*kref*tof

[0019] in:

[0020] E1; E2 – Energy for each scan of receiver A or B,

[0021] T = tA = tB = ti – maximum flight time = 30ns, for a maximum distance of 4.5m (uniqueness range),

[0022] kref – reflection coefficient of the target,

[0023] tof – the time of flight of the pulse,

[0024] P(R) – the reflected pulse power when kref = 1,

[0025] The amount of charge is proportional to the corresponding energy value of each scanning process.

[0026] Simplified, the following can be written for 200 accumulations:

[0027] in

[0028]

[0029]

[0030] A, B – the amount of charge accumulated in the storage capacitor A or A,

[0031] p – is the proportionality coefficient, its specific characteristics are irrelevant to these considerations.

[0032] By normalizing (dividing by the sum of the charges accumulated in the two capacitors), normalized charges Q1 and Q2 that are independent of reflectivity can be formed.

[0033] Q1 = A / (A+B) (eliminating the dependence on reflectivity by forming a quotient)

[0034] Q2=B / (A+B)

[0035] The values ​​for the flight time can be derived from the normalized charge for each capacitor separately:

[0036] tof1=(1-Q1)·T

[0037] tof2=Q2·T

[0038] In the ideal case without errors, the two flight times tof1 and tof2 determined in this way are identical: tof1=tof2. Possible measurement errors can be avoided by forming a mean value:

[0039] tof = 1 / 2 (tof 1 + tof 2) (average value)

[0040] The distance measured between the sensor and the reflecting object is:

[0041] S=1 / 2(c*tof)c–speed of light

[0042] An extended variant of the described structure consists in that each receiver has two storage capacitors (receiver A has Aa and Ab; receiver B has Ba and Bb). This property results in extended functional possibilities.

[0043] First possibility:

[0044] Receivers A and B operate identically. Storage capacitors Aa and Ab are switched in a staggered manner - Ab takes over the function of B (as described above).

[0045] Instead of two photoreceivers, only one photoreceiver is used, each with two storage capacitors:

[0046]

[0047]

[0048] With the right structure, this arrangement can have the advantage of a higher density of receivers (better resolution).

[0049] Second possibility:

[0050] Additional storage capacitors are used for the four-phase scanning of the signal. Figure 1 The signal scan with two integration time periods (two phases) is described in , but now the signal can be scanned four times, where the scanning points are staggered by 1 / 4 of the signal period, see Figure 2 In an implementation variant, a periodic pulse sequence with a period duration T can also be used as the transmission signal.

[0051] The phase shift of the reflected signal due to the flight time tof It can be calculated as follows from the charge amounts Aa, Ab, Ba, and Bb accumulated in the capacitors.

[0052]

[0053] Strictly speaking, the calculation of the phase angle is only applicable to sinusoidal signals.

[0054] Then, the phase shift To calculate the distance S between the sensor and the reflecting object:

[0055]

[0056] T – in this case the period of the periodic signal.

[0057] If a sinusoidal form is chosen as the periodic signal, the relationships shown for the second possibility also apply.

[0058] Which method is appropriate depends largely on the specific application and system design of the TOF sensor.

[0059] Figure 1 The method described in is usually used with very powerful pulses at a low duty cycle (e.g., >25W pulse power; 1:1000), while Figure 2 The pulse sequence shown in or with a 1:1 duty cycle is applied at power levels of 1 W to 4 W. For sinusoidal modulation of the light source, the choice of peak power is similar. The limits on the power used must be observed by maintaining eye safety.

[0060] Use of Active Pixel Sensor (APS)

[0061] It is known to provide an active pixel sensor (APS) for each image point. An active pixel sensor generally has a photodiode and a plurality of (field effect) transistors. In the cutoff layer of the photodiode, the light falling on the photodiode releases the charge, so that the (cutoff) voltage across the diode decreases due to the incident photons. The active pixel sensor operates as follows: At the beginning of the activation period, the voltage across the photodiode is set to a defined initial value by means of a reset transistor. During the activation period, the cutoff layer capacitance of the photodiode is discharged by the photocurrent caused by the incident reflected light pulse. In this case, the voltage across the photodiode decreases in proportion to the intensity of the reflected light pulse and to the duration of the part of the light pulse that falls within the activation period of the active pixel sensor. At the end of the activation period, the voltage value of the voltage dropped across the photodiode is read and fed to an analog post-processing or immediately to an analog-to-digital converter (ADU). For this purpose, each picture element has an amplifier transistor, which is switched column by column by means of a selection transistor to a common readout line for all picture elements in a row. Active pixel sensors can be implemented in CMOS (Complementary Metal-Oxide-Semiconductor) technology. For example, since the charge in the cutoff layer of the photodiode can also be released by thermal treatment, this usually results in signal noise superimposed on the actual photoelectric signal.

[0062] Correlated Double Sampling (CDS)

[0063] In order to avoid the above-mentioned signal noise, it is known to measure the voltage on the photodiode not only once at the end of the activation time period, but also to measure the voltage on the photodiode again subsequently after resetting the photodiode to restore the full cutoff voltage (dark voltage), in order to obtain a reference signal with which the noise signal component can be at least partially compensated.

[0064] A sensor implementing such correlated double sampling has a corresponding circuit for each pixel, which is referred to herein as a CDS stage. The CDS stage can be implemented, for example, by means of an operational amplifier.

[0065] Pixel structure

[0066] A pixel of an image sensor can have the following components:

[0067] - Active Pixel Sensor (also referred to herein as APS cell)

[0068] - a CDS level, which is associated with the APS unit, and

[0069] - Sample and Hold stage (S&H), used to read the pixels. Summary of the invention

[0070] The object of the present invention is to design the structure of the TOF sensor described above so that the probability of a dangerous failure is extremely low and can preferably only occur with a probability of less than 1 in 1100 years (according to ISO 13849-1 SIL3 or Category 4, PLe).

[0071] According to the present invention, in order to achieve this purpose, a 3D-TOF having at least one pixel matrix is ​​proposed, which has a pixel structure divided into at least two signal paths (channels A and B), wherein the signal paths are guided so that corresponding to the pixel positioning, the signal propagation time is the same for both channels (H-tree), and wherein each signal path has its own independent analog signal output (signal A and signal B), and wherein the 3D-TOF sensor also has at least two optical emitter groups, wherein each emitter group contains at least one emitter.

[0072] The emitter is preferably designed to emit in the form of infrared light, so that it operates in the infrared spectral range of light.

[0073] Preferably, each signal path of the pixel matrix is ​​provided with its own logic circuit (logic circuit of signal path A and logic circuit of signal path B) for control. Preferably, the logic circuit of each signal path is constructed to determine the amplitude and distance value by processing the signal values ​​of the two channels in a cross-exchange channel data manner.

[0074] According to a first inventive variant, the photosensitive surfaces of a pixel of the first channel (pixel A) and a pixel of the second channel (pixel B) are optically connected so that both pixels always receive the same amount of light.

[0075] The photosensitive surfaces of the pixels of the first channel (pixel A) and the pixels of the second channel (pixel B) can be connected by means of a diffuser, which evenly distributes the amount of light on the optical surfaces of the individual pixels from channel A (pixel A) and the individual pixels from channel B (pixel B). This method is referred to herein as optical binning. Here, the signal paths of the pixel groups, which respectively include pixels A and pixels B, are not electrically connected to one another. In one application case, the pixel matrix can be read by addressing the pixels in channel A and channel B in parallel. Therefore, at the output of the pixel matrix, there are always signal values ​​of signal A1 and signal B1; signal values ​​of signal A2 and signal B2; ...; signal values ​​of signal An and signal Bn.

[0076] Preferably, a comparator stage is provided, which is designed to compare the signal values ​​present in parallel at the output of the pixel matrix with one another. In this case, the digitized output value of the comparator is preferably compared with a predetermined value in at least one of the logic circuits A or B. If the two signal values ​​then have a large difference and thus, for example, exceed a comparator comparison value for the difference, the logic circuit preferably generates a failure signal and the reading process of the pixels of the 3D-TOF sensor is aborted.

[0077] Preferably, an analog-to-digital converter (ADU) is provided, which is associated with channel A or channel B and is constructed to digitize the analog signal values ​​A and B in the corresponding channel and feed them to the logic circuit of channel A or the logic circuit of channel B, respectively. Subsequently, the digitized channel values ​​can be compared. For this purpose, the channel data are preferably cross-exchanged. If the calculated difference exceeds the comparison value, the logic circuit preferably generates a failure signal and aborts the reading process.

[0078] Preferably, the 3D-TOF sensor is constructed for performing an additional special signal processing step, more precisely, for checking the functionality of the photoreceiver by comparing the signal value A and the signal value B in an additional special signal processing step. With the activation of the emitters (of at least one emitter group), the pixel A and the pixel B are activated simultaneously, more precisely, the storage capacitors of the respective pixels are activated. For the two-phase mode, the activation period corresponds to twice the length of the light pulse duration, and in the four-phase mode, the activation period corresponds exactly to the length of the light pulse duration. By means of this signal step, at the output of the pixel matrix with optically binned pixels, the same signal values ​​are expected at the output of the two channels A and B, respectively. The same signal values ​​(below the comparator threshold) are an indication for the functionality of the pixel.

[0079] Preferably, the 3D-TOF sensor is constructed for performing a further additional signal processing step in which the functionality of the emitter group is checked. With the activation of the emitters (of only one emitter group), the pixel A or the pixel B is activated, more precisely, the corresponding storage capacitor of the respective pixel is activated. The activation time period is implemented with twice the activation duration relative to the image period. For the two-stage mode, the duration of the activation time period corresponds to twice the duration of the light pulse, and in the four-stage mode, the duration of the activation time period corresponds exactly to the duration of the light pulse. As a result of this further additional signal step, at the output of the pixel matrix with optically binned pixels, different signal values ​​are expected at the output of the two channels A and B, respectively.

[0080] The difference in the signal values ​​at the output of the analog pixel matrix is ​​an indication that the corresponding emitter group has emitted a light pulse and is functional.

[0081] If verification is provided for one emitter group, the same method can be used for the other emitter group.A preferred method is to alternate the emitter groups accordingly between two image periods.

[0082] Preferably, the 3D-TOF sensor is designed such that the start of an image cycle is accompanied by a check of the first emitter group (claim 10), and such that a check of the photoelectric receivers is then carried out (claim 9), and then the actual image processing follows. If an image cycle is completed, the next image cycle is started with the next (i.e. at least the second) emitter group. This alternation of emitter groups is carried out as desired.

[0083] According to a variant embodiment, the 3D-TOF sensor can be constructed to omit the signal step of checking the first emitter group (claim 10) by the fact that the emitter groups operate in at least two different wavelength ranges (which may overlap if necessary) and the pixel matrix is ​​constructed so that pixels A and pixels B are determined in one of the two different wavelength ranges by corresponding optical filters in that the corresponding optical filters only allow one of the two wavelength ranges to pass. A particularly preferred solution is to use a wavelength of 850 nm for the pixels of the first emitter group and channel A and a wavelength of 940 nm for the pixels of the second emitter group and channel B. If the signal step according to claim 7 is now implemented, in this case the first and second emitter groups are activated simultaneously. The pixels of channel A receive reflected light pulses in the first wavelength range (850 nm) and the pixels of channel B receive reflected light pulses in the second wavelength range (940 nm). If, for the respective adjacently addressed pixels, signal values ​​of equal magnitude or signal values ​​having a predetermined ratio are then present at the outputs of channel A and channel B, this can prove that the addressed photoreceivers and the two emitter groups of both channels are functional.

[0084] Preferably, optical binning can also be provided after the two wavelengths in a supplementary manner to improve the consistency of the signal values ​​of the above optical filtering.

[0085] Preferably, the 3D-TOF sensor can be improved by adjusting the area of ​​the photoelectric receivers of channel A and channel B corresponding to the selected wavelength, thereby compensating for the different quantum efficiencies for different wavelengths, that is, in a complete 3D-TOF sensor, there are signal values ​​of the same size at the output of channels A and B.

[0086] Preferably, each channel has its own processor, which is connected to the signal path of the logic circuit of the channel. In addition, the processor is preferably connected to the input of each other channel. Therefore, the calculated amplitude and distance values ​​that have been transmitted by the corresponding logic circuit are transmitted to each processor.

[0087] Preferably, the logic operation of each channel logic circuit is checked in the following manner: each processor calculates at least one check value (random number) completely independently of the other processor. Typically, one check value is calculated for the two-stage mode and two check values ​​are calculated for the four-stage mode. Each check value is transmitted to the logic circuit of the same channel by the processor. The logic circuit uses the transmitted check value and the check value of another channel to calculate at least one output value, preferably two output values. The operation for calculating the output value is the same as the operation for calculating the amplitude and distance value. The output value is transmitted to the processor of the corresponding channel. The processor calculates the expected output value on his side and checks the output value obtained from the logic circuit of its channel against the output value calculated by itself, and outputs a failure signal in the case of deviation.

[0088] In a particularly preferred embodiment, one of the two processors is preferably a safety-related processor. The safety-related processor has at least two independent, non-interacting computing cores and processes two inputs into one output (1oo2 - one out of two). Preferably, the architecture is used to separate the image operations into a safety image processing process and a non-safety image processing process. The safety process only checks whether a deviation of a single image point from the expected standard position (deformation of the 3D point cloud) occurs in the defined field of view of the 3D-TOF sensor. If such a deviation occurs, an alarm signal is output.

[0089] Complex image processing is performed on the non-secure processor in order to identify the actual cause of the point cloud deformation (e.g., identifying the movement of a person, hand, head, etc.).

[0090] According to the second invention variant, a 3D-TOF sensor is proposed, which is preferably characterized by the following features:

[0091] The 3D-TOF sensor has an image recorder with a pixel matrix, in which

[0092] The pixel matrix has at least two separate independent blocks and transmits data in at least two separate signal paths; and / or

[0093] The pixels of the pixel matrix of the image recorder have at least a photosensitive receiver and a storage unit for storing charge carriers; and / or

[0094] The signal path is composed of at least an analog-to-digital converter (ADU), a programmable logic circuit, a storage unit and a processor connected thereto; and / or

[0095] The image recorder has at least a global shutter for all blocks, which can be triggered by a control signal of a programmable logic circuit and can set a minimum shutter time of less than 100ns; and / or

[0096] Signal paths are routed independently of each other and without interaction; and / or

[0097] Signal paths are routed so that data can be exchanged across logic circuits or processors; and / or

[0098] Each block of the image recorder is associated with its own imaging optics, so that the same image of the object side is projected onto the block. The distance of the optics is chosen to be so small that parallax errors cannot be resolved; and / or

[0099] Each block is associated with at least one illumination source that is synchronized with the global shutter via logic circuitry.

[0100] Preferably, the image recorder is characterized in that each block has an individual global shutter which can be controlled by the logic circuit of the associated signal path.

[0101] Preferably, the image recorder is characterized in that the global shutter of the block depends only on a single logic signal of a single signal path and that its dependency is set by a settable delay time (register value).

[0102] Alternatively or additionally, the image recorder is also preferably characterized in that for each block

[0103] associated with a narrow spectral range by means of an optical filter or optical filter system, and / or

[0104] The optical filter system is formed by a filter directly on the image recorder and at least one further filter in the imaging system of the optical device; and / or

[0105] The optical filters on the blocks of the image recorder act as edge filters ensuring a clear spectral separation of the blocks, while the filters in the optical imaging system are designed as band filters and include the spectral range of the blocks, and / or

[0106] Each spectral range of a block is respectively associated with an illumination source which emits light at exactly the wavelength corresponding to the transmission range of the optical filter of the respective block.

[0107] This corresponds to a method for reliably recording 2D image data with an image recorder of the type described above, which has the following method steps:

[0108] In a first processing step, the global shutter is triggered with a predetermined exposure time. As a result, in the memory of the pixel, charge images corresponding to the object imaging are respectively present on two blocks.

[0109] In a second processing step, the image data of the first block is transferred to the processing line of the first block (read block 1), and the image data is transferred to the processing line of the second block (read block 2), etc. As a result of this step, the image data now exists in digital form in the blocks.

[0110] The present invention will now be described in more detail with reference to the accompanying drawings based on embodiments.

[0111] Figure 1 A distance measurement according to the TOF principle is described, which is based on the measurement of the flight time of an emitted light pulse (transmit pulse);

[0112] Figure 2 Explains the Figure 1 Similar propagation time measurements using four-phase scanning of the reflected light pulses;

[0113] Figure 3 The basic structure of a TOF sensor according to the invention is described, which has two parallel paths and a cross comparison between them;

[0114] Figure 4 It shows how it can be achieved at the pixel level of a 3D-TOF sensor Figure 3 The basic structure shown in ;

[0115] Figure 5 a- Figure 5 c illustrates different variations of how the pixels of the pixel matrix can be divided into two channels (channel A and channel B);

[0116] Figure 6 a. Figure 6 b illustrates in cross-section how optical binning can be achieved;

[0117] Figure 7 by Figure 5 Taking the pixel matrix in a as an example, it is explained how the pixels in the pairs of channels A and B to be analyzed can be optically connected in parallel through optical binning;

[0118] Figure 8 illustrates the simultaneous scanning of the reflected light pulses;

[0119] Fig. 9 Shows that according to Figure 8 The principle shown in Figure 6 and Figure 7 Circuit diagram of a pixel matrix;

[0120] Fig.10 It is shown that the illumination for outputting light pulses is also designed doubly and can have two emitter groups with two emitters each, which are used alternately;

[0121] Fig.11 Explains the replacement Figure 8 The simultaneous scanning of pixels A and B shown in , only one of the pixels, namely pixel A or pixel B, is scanned in order to obtain a measure of the amount of light for the activated emitter group in this way;

[0122] Fig.12 a. Fig.12 b illustrates that corresponding pixels A and B of the pixel matrix may be sensitive to different wavelengths or different wavelength ranges;

[0123] Fig.13 It is shown that if the quantum efficiency of a pixel should be different for the corresponding wavelength, then according to Fig.12 a or Fig.12 The pixels in the pixel matrix of b may have areas of different sizes in order to obtain the results shown with comparable levels of output signals by compensating the lower quantum efficiency by the larger pixel area;

[0124] Fig.14 A logic circuit with an additional comparator is shown for testing channels A and B operating in two wavelength ranges respectively;

[0125] Fig.15 It is shown that, instead of two identical processors A and B, two different processors can be used to analyze the signals of channels A and B, one of the two different processors being a safety-related processor;

[0126] Fig.16 The schematic structure of the image sensor is shown, wherein, by expanding the circuit, the operational amplifier of the CDS level is also used as a comparator for comparing the signal values ​​of pixels A and B on the analog pixel layer;

[0127] Fig.17 The read cycle is explained;

[0128] Fig.18 shows the first switching state Fig.16 CDS level of image sensor;

[0129] Fig.19 shows the second switching state Fig.16 CDS level of image sensor;

[0130] Fig. 20shows the third switching state Fig.16 CDS level of image sensor;

[0131] Fig.21 An image recorder with high resolution is shown, which has a plurality of parallel readout channels for reading out the pixels block by block;

[0132] Fig. 22 An image recorder having two pixel blocks A and B is shown, each of which is sensitive to a different wavelength range;

[0133] Fig.23 It is shown that it is possible to use the same method on the block level as on the pixel level (compared to Figure 8 ) performs the check on the pixel matrix function in exactly the same way;

[0134] Fig.24 An image processing circuit for reading a pixel matrix divided into two blocks A and B is shown;

[0135] Fig.25 shows a similar Fig. 22 An image recorder having an additional optical channel arranged in the image recorder; and

[0136] Fig.26 The serial scanning of reflected light pulses by means of an image recorder having only a single global shutter is described.

[0137] The starting point for the invention is a general arrangement as described in EN ISO 13849 (Safety of Machinery). SIL-3 (Category 4; PLe) can then be achieved in such a way that the system with at least two signal paths is designed so that integrity can be achieved by cross-comparison of two parallel signal paths, see Figure 3 .

[0138] according to Figure 3 The 3D-TOF sensor 10 shown in FIG. 1 has two input units 12A and 12B, which can be formed by pixels. Figure 4 The output signals generated by the input units, i.e. the output signals of the pixels, for example, are respectively fed to logic circuits 14A and 14B, which analyze the output signals of the input units 12A and 12B. The logic circuits 14A and 14B are connected to one another for the purpose of cross comparison. Within the scope of the cross comparison, it is checked in particular whether the logic circuits 14A and 14B respectively provide the same output signals. If the cross comparison is positive, the signals acquired by the input units 12A and 12B and processed by the logic circuits 14A and 14B are output to the corresponding output units 16A and 16B.

[0139] Therefore, according to Figure 3 and Figure 4 The 3D-TOF sensor of each has two parallel signal paths, one of which is formed by the input unit 12A, the processing logic circuit 14A and the output unit 16A, and the second signal path is formed by the input unit 12B, the processing logic circuit 14B and the output unit 16B. Thus, the first signal path forms channel A and the second signal path forms channel B. Now it is proposed to apply this parallel system to a 3D-TOF sensor.

[0140] Therefore, in the following, the terms "signal path" and "channel" are used synonymously.

[0141] First Invention Variation

[0142] Within the scope of the first inventive variant, the above described principle with at least two pixels (pixel A and B) is applied so that a continuous parallel processing with comparison of signal paths A and B is achieved. The comparison of the signals is performed at different levels of signal processing, see Figure 4 .

[0143] All pixels A and B are arranged in the same rows and columns in a matrix form, or in a chessboard form.

[0144] The signal paths are implemented strictly separately as channel A and channel B. This applies to both the analog and digital signal paths, regardless of whether the receiver matrix integrated in silicon is implemented only analogically or both analogically and digitally, see Figure 5 .

[0145] There are therefore two implementation possibilities, namely that only the analog circuit parts are integrated in silicon and external logic circuits are used on the FPGA, or that all components are fully integrated in silicon or another semiconductor material.

[0146] Optical binning / check receiver functionality

[0147] Combining multiple pixels on a circuit layer is known in the art. This method is used to:

[0148] - Reduce the resolution,

[0149] - Increase image speed, or

[0150] - Improved sensitivity and reduced noise.

[0151] Optical binning is introduced as a new feature to ensure that pixels A and B receive the same optical power (imaging).

[0152] Optical binning can be achieved by arranging a homogenous diffuser on pixel A and pixel B. This arrangement can be realized in such a way that only the photosensitive surfaces (photoreceivers) involved in pixels A and B, respectively, are optically connected. However, the diffuser can also be placed on the entire pixel geometry. Additional variations are given by using micro-optics, as they have been used many times to increase the fill factor in CMOS image sensors. The micro-optics are then arranged on the diffuser.

[0153] In a further embodiment variant of the first inventive variant, the diffuser and the micro-optics can also be realized in one component on the basis of the micro-optics.

[0154] With respect to the matrix arrangement of pixels A and B, A1 and B1, A2 and B2, ... An and Bn are optically connected to each other, it is assumed that the analog signal paths of A1 and B1 or A2 and B2, An and Bn are implemented to maintain the same signal propagation time at the same length. This requirement also applies to the digital control signals of the switching transistors of the individual optically binned pixels, see Figure 6 and Figure 7 .

[0155] The functionality of the parallelized optoelectronic receivers is checked by an additional signal step. The purpose of this signal step is to check whether channels A and B deliver the same signal values. These signal values ​​must be significantly above the noise level (zero signal).

[0156] This is achieved by simultaneously scanning the reflected light pulses. For the two-phase mode, the scanning time corresponds to twice the length of the light pulse duration (see Figure 8 ), and in the four-stage mode, the scanning time corresponds exactly to the length of the light pulse duration. In this case, the cycle of emitting and scanning the reflected light pulse should preferably also be repeated many times until a sufficient amount of charge is accumulated in the memory of pixel A and pixel B.

[0157] Fig. 9 The circuit diagram is schematically illustrated in . After the integration phase sequence, the memories of the optically binned pixels (A1 and B2) are selected simultaneously and placed at the output of the analog matrix. The charge quantities of pixels A1 and B1 are compared for the first time in the analog signal path (for example by using a comparator). It is checked whether the difference voltage between A and B is below a previously defined threshold value. Depending on the number of memories present in the pixel (two-stage or four-stage), this cycle can be carried out once or twice per pixel. A circuit-technical parallelization of the memories a and b can also be achieved.

[0158] Further, the signal value is converted to an analog-to-digital converter (ADU A and ADU B; see Fig. 9 ) and perform a second comparison. This operation can be performed by the downstream logic circuit and / or by the processor A or B downstream of the corresponding logic circuit A or B. In terms of implementation, the logic circuit A and the logic circuit B are different from the processor A or the processor B. The logic circuit is a circuit that implements logical operations on the circuit layer and can exist in the form of an FPGA (field programmable gate array), for example. Conversely, the operating principle of the processor is predetermined by sequential programming (for example, in the form of firmware). For example, the processor can be a digital signal processor (DSP).

[0159] If a defined difference value is exceeded in the analog or digital range, a fail signal is triggered in the logic block and the image recording is stopped. The fail signal is transmitted directly to the processor interface (output) via a signal path.

[0160] The processor outputs connected to the safety control can then direct the machine to a safe level, see Fig. 9 .

[0161] Dual implementation of illumination source / check emitter functionality

[0162] Just as the signal path is designed doubly, the illumination of the 3D-TOF sensor with optically binned pixels is also designed at least doubly. At least two emitter groups are used alternately, see Fig.10 .

[0163] A transmitter group consists of at least one transmitter. Fig.10 , and are represented by two emitters in each case. Emitters 1 and 3 as well as emitters 2 and 4 are respectively operated in parallel and alternately. LEDs or laser diodes can be used as emitters (eg VCSELs), see Fig.10 .

[0164] Extended cycle for checking transmitter functionality

[0165] By means of optical binning and an additional signal step, it can be checked whether the optoelectronic receiver and the signal processing channel provide the same signal value. The identity of the signal values ​​can be determined by a difference value, which should be below a threshold value to be determined.

[0166] In this step, no conclusions can be drawn about the functionality of the transmitter. Further additional processing steps can provide information about this.

[0167] If not Figure 8, instead of scanning the reflected light pulses of pixel A and pixel B simultaneously as described in , only one pixel (i.e. either A or B) is scanned. When determining the difference, a signal above a threshold value is generated in the comparator stage (analog) or in the digital logic part, which threshold value provides a measure of the amount of light for the activated emitter group, see Fig.11 .

[0168] In the running image cycle, the corresponding Figure 8 and Fig.11 The signal step is placed before the actual image generation to monitor the signal path.

[0169] Check transmitter group 1 Check the receiver Image Processing Check transmitter group 2

[0170] Transmitter group 1 and transmitter group 2 are activated alternately in a cycle.

[0171] Alternative check mechanism for transmitter functionality by decomposing the transmitter wavelength

[0172] A particularly advantageous implementation of the continuous parallelization of the signal transmission can be achieved by having emitter groups emitting at different wavelengths, while pixels A and pixels B can each selectively react to the respective different wavelengths, see Fig.12 a and 12b.

[0173] The advantage of this method is that it is not necessary to iteratively refer to the first transmitter group and the second transmitter group once. Fig.11 The signaling steps described, rather, can be carried out simultaneously, since both transmitter groups can be activated simultaneously and can be selectively identified by the wave selectivity of the receiver.

[0174] Optical binning is thus done wavelength selectively. That is, a typical implementation might be to use narrowband filters and a homogenizing function with a central wavelength of 850 nm, and for a second receiver group with a central wavelength of 940 nm.

[0175] In a particular implementation, the photosensitive surface of the photoreceiver can be designed so that the different quantum efficiencies of the different wavelengths are compensated (in silicon, the quantum efficiency for 940 nm drops by about 30% relative to 850 nm). A larger sensitive area of ​​the receiver with low quantum sensitivity can compensate for this disadvantage, see Fig.13 .

[0176] Fig.12 Different arrangements of wave-selective solutions are shown. Fig.12 In a, A and B work at two wavelengths respectively. The functionality of A and B is checked for each wavelength. As a result, additional comparators can be used. Depending on the wavelength used, logic circuit A or logic circuit B processes the comparator signal ( Fig.14).

[0177] If you use the Fig.12 b) and in the check step the transmitter groups with two different wavelengths are activated simultaneously, then a transmitter check and a receiver check can be performed simultaneously. Where the difference signal at the comparator output between channel A and channel B is less than a defined threshold, it can be concluded that the two transmitter groups and the opto-receiver of the two channels are functioning properly.

[0178] Functions of logic circuits A and B

[0179] Logic circuit A provides control signals for the photoelectric receiver of channel A, i.e., for controlling Figures 18 to 20 1 is a diagram showing the switches of the CDS stage for signal path A. Logic circuit B provides control signals for the optoelectronic receivers of channel B, i.e., for controlling the Figures 18 to 20 The switches of the CDS stage for signal path B are shown in FIG. 1 . The synchronization of the logic circuits ensures that the correct pixel pair is addressed in each readout process. The validity of the signal value is determined by the comparator signal.

[0180] In a further processing step, the digital difference signal of the channel is calculated separately in each logic circuit. To this end, the logic circuit exchanges the digitized channel data as long as the digitized channel data is considered valid. If the digitized difference signal of pixels A and B exceeds a threshold value, a fail signal is transmitted to the corresponding processor.

[0181] The two logic circuits calculate the signal amplitude value and the distance value independently of each other. The calculated value of logic circuit A is transmitted to processor A, and the value of logic circuit B is transmitted to processor B. In addition, the calculated values ​​are respectively provided crosswise. Then, each processor implements a comparison of the amplitude value and the distance value. If the comparison results in a deviation, the corresponding processor provides a failure signal.

[0182] Functional testing of logic circuits

[0183] Between image cycles, the processor sends a digital input value (random number) to the upstream logic circuit. The logic circuit feeds this random number to its processing channel and calculates an output value based on it, which is checked for correctness by the processor routine of each processor. The processing process involves the exchange of input values ​​between the logic circuits. Therefore, the checking process of each logic circuit uses the input values ​​of channel A and channel B for calculation.

[0184] The calculation function corresponds to an operation for determining the distance value and the amplitude value. In the event that the comparison in the processor results in an error, a failure signal is output by the corresponding processor.

[0185] Functions of Processors A and B

[0186] In a first implementation, the two processors each implement the same operation. Each processor has its own memory and its own interface.

[0187] The outputs OUT A and OUT B are processed in a downstream layer (eg safety-related control).

[0188] In a second embodiment, which is particularly advantageous for complex image processing operations (eg analysis of human movements), a safety-related processor is used for one of the processors (eg channel A).

[0189] Safety-related processors have a parallelized, non-interacting architecture. Such processors are known (e.g., HICore 1 – https: / / www.hima.com / en / products-services / hicore-1 / ) and have, for example, a so-called 1oo2 structure (two out of one). In this case, the processing is divided into safety processes and non-safety processes.

[0190] The division is illustrated in the following example.

[0191] For monitoring the work area, it is important to prevent hands from entering the safety zone or to stop the machine as soon as hands enter the area.

[0192] Since identifying a hand requires certain complex image processing operations (computation time), the processing is divided into specific areas defined as 3D safety zones. As a result, the safety-related processor only controls the position of the point cloud within the safety zone defined in the FOV (field of view). Deviations (deformations) of the point cloud will result in a failure signal. Monitoring the position of the point cloud is a relatively simple operation.

[0193] A further processor checks which object has entered the safety zone. The corresponding calculations are complex and are implemented using modern high-level languages. If it is a hand, an indication is given accordingly, for example to the operator, see Fig.15 .

[0194] Furthermore, there is the possibility of implementing the operational amplifier of the CDS stage as a comparator by expanding the circuit, in order to compare the signal values ​​of the pixel A and the pixel B on the analog pixel level.

[0195] Fig.16 The schematic structure of the corresponding image sensor is shown in the figure below. Figures 18 to 20 , the CDS stages in different switching states are shown in more detail.

[0196] For example, here a pixel consists of an APS cell, a CDS stage and a sample and hold stage (labeled S&H in the drawing).

[0197] On the one hand, the CDS stage serves as an accumulator for the necessary multiple exposures and at the same time as a stage for suppressing noise components from the reset of the photodiodes of the APS unit and for separating the background light.

[0198] Now, a cross comparison with neighboring pixels according to the sequence already explained is fed directly into the CDS stage.

[0199] Fig.17 Describes the read cycle.

[0200] Using CDS-level functions, it is easy to explain how things work.

[0201] Phase 1:

[0202] In pixel A—switches S1a, S3a are closed; switches S2a, S4a and S5a are open.

[0203] In pixel B—switches S1b, S3b are closed; switches S2b, S4b and S5b are open.

[0204] The following ratio is obtained:

[0205] In pixel A:

[0206] Q′1a=C1a(Va-Vref); Q′2a=C2a(Vref-Vouta(n))

[0207] In pixel B:

[0208] Q'1b=C1b(Vb-Vref); Q'2b=C2b(Vref-Voutb(n))

[0209] Fig.18 Pixel A and pixel B are shown in a first switching state of the CDS stage in phase 1 .

[0210] Phase 2:

[0211] In pixel A—switches S2a, S4a are closed; switches S1a, S3a and S5a are open.

[0212] In pixel B—switches S2b, S4b are closed; switches S1b, S3b and S5b are open.

[0213] In pixel A:

[0214] Q″1a=C1a(Vb-Vref); Q″2a=C2a(Vref-Vouta(n+1))

[0215] In pixel B:

[0216] Q″1b=C1b(Va-Vref); Q″2b=C2b(Vref-Voutb(n+1))

[0217] Then using the node rules, it can be written as:

[0218] In pixel A:

[0219] Q′1a+Q′2a=Q″1a+Q″2a

[0220] If we insert the relation for the charge into this equation, we get the following after the corresponding transformations:

[0221] Vouta(n+1)=C1a(Vb–Va)+C2aVout(n)

[0222] Equivalently, the ratio at the output of pixel B is:

[0223] Voutb(n+1)=C1b(Va–Vb)+C2bVout(n)

[0224] In order to obtain a sufficient output voltage on capacitor C2a or C2b, phase 1 and phase 2 are repeated multiple times (N times).

[0225] Fig.19 Pixel A and pixel B are shown in a second switching state of the CDS level in phase 2.

[0226] Phase 3 – Inspection Phase

[0227] All switches are open, only switches Sa5 and Sb5 are closed. At this stage, the operational amplifier acts as a comparator in both pixels.

[0228] In this case, the voltages formed across the capacitors Ca2 and Cb2 are each compared with a reference voltage.

[0229] Fig. 20 Pixel A and pixel B are shown in the third switching state of the CDS stage in phase 3.

[0230] The capacitor Ca2 is applied with the differential voltage Vb-Va accumulated multiple times.

[0231] The capacitor Cb2 is applied with the differential voltage Va-Vb accumulated multiple times.

[0232] Then, compare the three situations.

[0233] 1. N×Vouta(n+1)=N×Voutb(n+1)—This would be an ideal situation if pixel A and pixel B provided the same value. Due to the physical properties of real components, this ideal situation is only a theoretical feature.

[0234] 2. N×Vouta(n + 1) ≈ N×Voutb(n + 1) – The values at the outputs of Pixel A and Pixel B are approximately the same. One of the values is slightly higher or lower than the other. In this case, both pixels can be considered to be functioning properly.

[0235] Logical 0 appears on both output lines of the comparator.

[0236] 3. N×Vouta(n + 1) ≠ N×Voutb(n + 1) – The output value of Pixel A is much larger or much smaller than the output value of Pixel B. The pixel is considered to be damaged.

[0237] In this case, if Vb >> Va, then a logical 1 will be applied at the output of Pixel A.

[0238] In the case of Vb << Va, a logical 1 will be applied at the output of Pixel B.

[0239] After the comparison, the CDS stage will be reset and a new cycle can begin.

[0240] The control of the switches for the corresponding CDS stages is carried out by Logic Circuit A or Logic Circuit B associated with the corresponding signal paths.

[0241] Additional operating mode:

[0242] The introduced formation of the differences Vb - Va and Va - Vb can also be very advantageously applied in 4 - stage operation. In this case, the formation of the differences can be used to calculate the phase angle without reading individual values, as described further above, in such a way that the phase shift is calculated as follows

[0243]

[0244] Provided that two additional pixels are used, such that a set of four pixels or a set of two pixels but with a total of four memory cells forms a complete 4 - stage architecture.

[0245] Second invention variant

[0246] According to the second invention variant, the image recorder (imager) is configured, for example, in CMOS technology with high resolution, preferably having a plurality of parallel read channels, see Fig.21 . Despite the high resolution, the purpose of this architecture is to achieve a high frame rate. The read channels typically combine multiple rows or multiple columns of the image recorder matrix. This combination is further referred to as a block or as an imager block, see Fig. 22 .

[0247] Specifically, Fig. 22 It is shown that:

[0248] 1 – Image recorder,

[0249] 2 – Photosensitive surface,

[0250] 4 – Imager block A with spectral filter A,

[0251] 5 – Imager block B with spectral filter B,

[0252] 7 – Read channel for imager block A,

[0253] 8 – Read channel for imager block B.

[0254] The second inventive variant exploits this division of the image field into read blocks.

[0255] Further inventive variants are explained in more detail below based on the first exemplary embodiment.

[0256] Corresponding to the created readout channels, the image field, preferably with a very high resolution (eg 1280×1024 image points), is divided into blocks. Each readout block works with an individually triggerable global shutter.

[0257] Each block is further associated with its own imaging optics. The distance between the optics is so small that even at small object distances on the image side, parallax is no longer resolved. That is, the same image of the object side is generated on each block.

[0258] This arrangement consisting of optics and image recorder is used in accordance with Figure 4 In the parallelized reading circuit and image processing circuit.

[0259] Depth image calculation:

[0260] This embodiment is based on a two-channel structure. The image recorder (imager) is divided into blocks A and B. In the current embodiment, each block has 640×1024 image points. The digital read channels for blocks A and B are directed to the corresponding logic circuits A and B. It does not matter whether these logic circuit blocks already contain part of an integrated circuit or whether they contain external logic circuits (e.g. FPGA).

[0261] Logic circuits A and B control the corresponding global shutter signals and, correspondingly, also the illuminations LA and LB. The illumination sources operate synchronously. At time t0, for duration ti=tA, both sources are triggered simultaneously with the global shutter signal of block A. The light pulse is reflected on the target and recorded by all receivers A of block A with a staggered flight time tof. The charge quantity (photoelectrons) formed by the recording of the light pulse is stored on a storage capacitor (A1) belonging to each receiver. The receivers of block A can only record a portion of the reflected light pulse that corresponds proportionally to the time difference tA-tof.

[0262] The global shutter of the receivers of block B is activated with a time delay t0+tA. Receiver B is active for time tB (tA=tB in the example). The receiver also records only a portion of the reflected light pulse, which corresponds proportionally to the time difference tof+ti-tA (tof if ti=tA). The corresponding charge quantity is stored on the storage capacitor B1 of each receiver belonging to block B.

[0263] Since the reflected signal may be very weak depending on the reflectivity and distance of the target, an amplification mechanism must be provided. This amplification mechanism is achieved by repeating the above-described measurement several times and accumulating the corresponding recorded charge on the associated storage capacitor until a sufficient signal level is reached. For example, this process can be repeated 100 times.

[0264] The result is that for the time period t0 to tA, an image is present in the storage capacitor A1 of block A, while for the time period tA to tB, an image is present in the storage capacitor B1.

[0265] The two images are transmitted to the corresponding logic circuit via the corresponding read channel. Subsequently, the logic circuit cross-exchanges the image information according to the above rules and calculates the depth image pixel by pixel separately:

[0266] Q1 = A / (A+B) (eliminating the dependence on reflectivity by forming a quotient)

[0267] Q2=B / (A+B)

[0268] tof1=(1–Q1)*T

[0269] tof2=Q2*T

[0270] tof = 1 / 2 (TOF 1 + TOF 2) (average value)

[0271] S = 1 / 2 (c*tof)

[0272] Here, a calculation rule is applied so that the pixel is always calculated corresponding to its position in the receiver matrix. That is, according to the selected arrangement, A0,0 is with B0,0; A0,1 is with B0,1; ... Am,n is with Bm,n:

[0273] A0,0…A0,n

[0274] …

[0275] Am,0…Am,n

[0276] B0,0…B0,n

[0277] …

[0278] Bm,0…Bm,n

[0279] At the end of the calculation, both logic circuits contain a distance image each. Apart from the temperature-dependent calibration factor which can be determined once, the two distance images should be identical.

[0280] Check the functionality by comparing the depth images:

[0281] In a further operation, the consistency of the two depth images A and B is checked by a pixel-by-pixel comparison. This comparison can be performed on two logic circuits or on a downstream processor.

[0282] A threshold can be defined for the comparison, which specifies how many pixel pairs are allowed to deviate from the defined expected comparison value. The threshold defines whether a pixel pair provides the same value and what is considered equal or unequal. Furthermore, the number of unequal pairs can be used as a decision parameter.

[0283] If a defined threshold is exceeded on at least one processor, at least one of the processors generates a fail signal at the output.

[0284] The comparison of depth images A and B already accumulates several comparison functions. Pixel sensitivity, gate functions (integrate / accumulate), readout, ADU and logic operations all belong to this.

[0285] The full functionality of the pixel matrix can also be achieved through pixel-by-pixel comparison of the magnitude images.

[0286] Function to inspect the pixel matrix in a 2D magnitude image:

[0287] As on the pixel level (see further above and Figure 8 and Fig. 9 ), it is also possible to perform checks on the functions of the pixel matrix in the 2D magnitude image at the block level, see Fig.23 and Fig.24This is achieved by scanning the reflected light pulses in block A and in block B simultaneously. In this case, the cycle of emitting and scanning the reflected light pulses must also be repeated several times until a sufficient amount of charge is accumulated in the memories of the pixels of block A and block B.

[0288] After the accumulation cycle is complete, the images in blocks A and B are read and compared pixel by pixel in two logic circuits. The comparison values ​​can be evaluated both in the logic circuits and in the downstream processor.

[0289] A threshold value can be defined for the comparison, which determines whether the pixels must be considered equal or unequal. Furthermore, it can be defined how many pairs of pixels are allowed to deviate from each other (not equal) by a defined range before a fail signal is generated. If the defined threshold value is exceeded on at least one channel, a fail signal is generated at the output by at least one of the processors.

[0290] Check the lighting source:

[0291] The described inspection cycle is also suitable for inspecting the illumination sources. In the embodiment, a total of 4 illumination sources are used. In the embodiment, two of each are controlled by the logic circuit of channel A or channel B.

[0292] The illumination source can now be checked by using either the source of channel A or the source of channel B in each check cycle.

[0293] Rather than comparing receiver signal values ​​by cross comparison, comparison of illumination sources is performed by serial comparison among corresponding channels.

[0294] In this embodiment, the inspection process is performed as follows:

[0295] 1. Start the light pulse with twice the length compared to the depth image recording of the illumination source of channel A (e.g. 60ns).

[0296] 2. Start the global shutter in both blocks A and B simultaneously with twice the light pulse length (eg 120ns).

[0297] 3. The image of block A (image A0) and the image of block B (image B0) are stored in the logic circuit blocks of the corresponding channels.

[0298] 4. Start the light pulse with twice the length compared to the depth image recording of the illumination source of channel B (eg 60 ns).

[0299] 5. Start the global shutter in both blocks A and B simultaneously with twice the light pulse length (eg 120ns).

[0300] 6. Store the image of block A (image A1) and the image of block B (image B1) in the logic circuit blocks of the corresponding channels.

[0301] 7. Each pixel value A(i, j) from the A submatrix and each pixel B(i, j) from the B submatrix when exposed with the LA source is compared with the corresponding value when exposed with the LB source. A maximum difference value is defined. If this value is exceeded, there is damage and a failure signal can be triggered. It is also conceivable that a failure signal is only triggered if a predetermined number of pixel pairs exceeds the difference value. The comparison value can be evaluated both in the logic circuit and on a downstream processor.

[0302] The present invention will be described based on the second embodiment.

[0303] As in the first embodiment, the image field, preferably with a very high resolution (e.g. 1280×1024 image points), is divided into blocks corresponding to the created readout channels. Unlike the first embodiment, each readout block is subdivided again. That is, each readout block works with two individually triggerable global shutters.

[0304] Each sub-block (Aa, Ab, Ba, Bb) is further associated with its own imaging optics. The distances between the four optics are now so small that parallax is no longer resolved even at small object distances on the image side. That is, the same image on the object side is generated on each sub-block.

[0305] The arrangement consisting of the optical device and the imager is used in accordance with Fig.24 In the parallelized reading circuit and image processing circuit.

[0306] Depth image calculation:

[0307] This embodiment is also based on a two-channel structure. However, the image recorder (imager) is now divided into blocks Aa and Ab and blocks Ba and Bb. In this embodiment, each block has 640×512 image points. The digital read channels for blocks A and B are directed to corresponding logic circuits A and B.

[0308] Logic circuits A and B control the respective global shutter signals and correspondingly also control illuminations LA and LB. The illumination sources operate synchronously.

[0309] Although the first embodiment describes the signal scanning with two integration time periods (2 phases), the signal is now scanned four times, where the scanning points are respectively shifted by 1 / 4 period of the signal.

[0310] With regard to the embodiment with light pulses of 30 ns length, this results in a scanning sequence which can be described as follows: 0 to 30 ns for sub-block Aa, 30 ns to 60 ns for sub-block Ba, 60 ns to 90 ns for sub-block Ab, and 90 ns to 120 ns for sub-block Bb, see also Figure 2 .

[0311] If a sinusoidal form is chosen as the periodic signal, the shown relations also apply.

[0312] Since the reflected signal may be very weak depending on the reflectivity and distance of the target, the amplification mechanism described above must be set. This amplification mechanism is achieved by repeatedly measuring and accumulating the corresponding recorded charge on the associated storage capacitors of the pixels in the sub-block until a sufficient signal level is reached. For example, this process can be repeated 100 times.

[0313] As a result, in the storage capacitance of the pixels of the subblock there is one image for each quarter of the period. From this, the phase and thus the distance S of each pixel can be calculated according to the following rule.

[0314]

[0315]

[0316] T – in this case the period of the periodic signal.

[0317] Strictly speaking, the calculation of the phase angle is only applicable to sinusoidal signals. In the case of square wave signals, a corresponding correction must be applied for the high linearity of the distance characteristic. In addition, by scanning with T / 2, the possibility of improving the linearity of the distance characteristic in 4-stage scanning is realized.

[0318] The inspection process described in the first embodiment is carried out in the same manner.

[0319] Further, we will explain Fig.26 The third embodiment is shown.

[0320] Unlike the previous examples, only one global shutter is provided for the entire pixel matrix. Although in the previous embodiments, two global shutters were used for the phase of the depth image (see Figure 1 , Figure 8 , Fig.11 and Fig.23 ), so that two phase images can be recorded simultaneously (in parallel), but Fig.26 The embodiment shown in is based on an image recorder with only a single global shutter. In this case, the two phase images must be recorded one after the other (serially), e.g. Fig.26As described. Apart from this, the same optical structure as in the first embodiment can be set. The pixel matrix is ​​divided into two blocks A and B along the two reading channels. Each block is associated with its own imaging optical device. The distance between the two optical devices is very small, so that even in the case of small object distances on the image side, parallax is no longer resolved. That is, the same image on the object side is generated on each block.

[0321] In contrast to the first embodiment, each block additionally contains an optical filter or a system of filters. The filter or filter system limits the wavelength of the transmission of each block to a narrow spectral band corresponding to the source wavelength of the associated light source, see Fig.25 .

[0322] Fig.25 The examples in Fig. 22 The embodiment of is different in that an additional optical channel 3 and an additional reading channel 6 are provided. Specifically, Fig.25 It shows:

[0323] 1 – Image recorder,

[0324] 2 – Photosensitive surface,

[0325] 3 – Additional optical channels,

[0326] 4 – Imager block A with spectral filter A,

[0327] 5 – Imager block B with spectral filter B,

[0328] 6 – additional read channel,

[0329] 7 – Read channel for imager block A,

[0330] 8 – Read channel for imager block B.

[0331] For all the embodiments shown, it is applicable that the corresponding 3D-TOF sensor can be constructed so that the step of checking the signals of the first emitter group is omitted by the emitter groups operating in at least two different wavelength ranges (which may overlap if necessary) and the pixel matrix is ​​constructed so that the pixels A, B and B (or blocks A and B) are determined in one of the two different wavelength ranges by the corresponding optical filters, in that the corresponding optical filters only allow one of the two wavelength ranges to pass. A particularly preferred solution is to use a wavelength of 850 nm for the pixels of the first emitter group and channel A and a wavelength of 940 nm for the pixels of the second emitter group and channel B.

Claims

1. A 3D-TOF sensor, the 3D-TOF sensor having at least one pixel matrix, the pixel matrix having a pixel structure divided into at least two signal paths, wherein: at least two signal paths including a first channel A and a second channel B, wherein the signal paths are directed such that corresponding to the pixel positioning, the signal propagation time is the same for both channels, and wherein each signal path has its own independent analog signal output, The 3D-TOF sensor further comprises at least two optical emitter groups, each of which comprises at least one emitter. wherein the photosensitive surfaces of pixel A of the first channel A and pixel B of the second channel B are optically connected in pairs via optical binning so that both pixels receive the same amount of light, and wherein the signal paths of the pixel groups respectively including pixel A and pixel B are not electrically connected to each other.

2. The 3D-TOF sensor according to claim 1, wherein: The emitter operates in the infrared spectral range.

3. The 3D-TOF sensor according to claim 1 or 2, wherein: A logic circuit is provided for each signal path of the pixel matrix, wherein a logic circuit of signal path A is provided for signal path A and a logic circuit of signal path B is provided for signal path B, so as to control and process signal values ​​originating from pixels of the corresponding signal path.

4. The 3D-TOF sensor according to claim 3, wherein: The logic circuit of each signal path is constructed to determine the amplitude and distance values ​​by processing the signal values ​​of the two signal paths in a cross-exchanged manner.

5. The 3D-TOF sensor according to claim 3, wherein: The photosensitive surfaces of the pixels A of the first channel A and the pixels B of the second channel B are connected by means of a diffuser which evenly distributes the amount of light on the optical surfaces of the individual pixels A from the first channel A and the individual pixels B from the second channel B.

6. A 3D-TOF sensor according to claim 1 or 2, wherein the 3D-TOF sensor has a comparator stage, which is constructed to compare the signal values ​​present in parallel at the output of the pixel matrix with each other and / or with a predetermined value as a comparator threshold.

7. The 3D-TOF sensor according to claim 1 or 2 has two analog-to-digital converters, which are constructed to digitize analog signal values ​​A and B in the corresponding signal channel A or signal channel B, and are respectively connected to the logic circuit of signal path A or the logic circuit of signal path B.

8. The 3D-TOF sensor according to claim 1 or 2 is constructed to check the functionality of the photoelectric receiver by comparing signal value A and signal value B in an additional special signal processing step in such a way that pixel A and pixel B are activated simultaneously by activation of the transmitter.

9. The 3D-TOF sensor according to claim 1 or 2, which is designed to check the functionality of the emitters by a further additional signal step by activating the pixel A or the pixel B by activating the emitters of only one emitter group.

10. The 3D-TOF sensor according to claim 1 or 2, characterized in that: Each signal path has a processor which is connected to the signal path of the logic circuit of the channel, wherein the calculated amplitude and distance values ​​which have been delivered by the corresponding logic circuit are transmitted to each processor.

11. The 3D-TOF sensor according to claim 10, wherein: One of the two processors is a safety-related processor which has at least two independent, non-interacting computing cores and processes two inputs into one output.

12. A 3D-TOF sensor, comprising an imaging sensor having an image recorder with a pixel matrix, wherein: The image recorder is capable of reading the image data in at least two separate independent blocks of the pixel matrix, respectively, and the data are transmitted in at least two separate signal paths, The pixels of the pixel matrix of the image recorder have at least a photosensitive receiver and a storage unit for storing charge carriers, and The signal path comprises an analog-to-digital converter, a programmable logic circuit, a storage unit and a processor connected thereto. The image recorder has at least a global shutter for all blocks, the global shutter can be triggered by a control signal of a programmable logic circuit, and a minimum shutter time of less than 100 ns can be set; and The signal paths are routed independently of each other and without interaction, and The signal paths are wired to enable data from logic circuits or processors to be exchanged across, In this case, each block of the image recorder is associated with its own imaging optics, so that the same image of the object side is projected onto at least two separate independent blocks of the pixel matrix.

13. The 3D-TOF sensor according to claim 12, characterized in that: Each block is associated with at least one illumination source that is synchronized with the global shutter via logic circuitry.

14. The 3D-TOF sensor according to claim 12 or 13, characterized in that: The sensor is characterized in that each block has an individual global shutter that can be controlled by the logic circuitry of the associated signal path.

Images