Distance measuring device and distance measuring method

Abstract

The present disclosure relates to a distance measuring device and a distance measuring method that are capable of suppressing blurring of a moving subject. Provided is a distance measuring device comprising a light emitting unit that emits irradiation light, and a light receiving unit that receives reflected light resulting from the irradiation light being reflected on an target object, wherein: the irradiation light has a waveform shape in which N (N is a natural number equal to or greater than 2) modulation frequencies are superimposed; and the light receiving unit separates a distance measurement signal obtained from the reflected light into each modulation frequency, and outputs distance information by integrating distance measurement results obtained from the separated distance measurement signal. The present disclosure can be applied, for example, to the distance measuring device of indirect ToF method.

Description

Distance measuring device and distance measuring method

[0001] The present disclosure relates to a distance measuring device and a distance measuring method, and more particularly to a distance measuring device and a distance measuring method that are capable of suppressing blurring of a moving subject.

[0002] The indirect ToF (Time of Flight) method is one of the measurement methods used by distance measuring devices to measure the distance to an object. Indirect ToF distance measuring devices measure the distance to an object by irradiating the object with light having a predetermined period and detecting the phase difference between the irradiated light and the reflected light.

[0003] In an indirect ToF distance measuring device, there is a trade-off between the distance measurement range and the distance measurement accuracy. Patent Document 1 proposes an indirect ToF distance measuring device that can achieve both a high distance measurement range and a high distance measurement accuracy by using two modulation frequencies, a first frequency and a second frequency.

[0004] Japanese Patent Application Publication No. 2020-134464

[0005] In an indirect ToF distance measuring device, for example, when two types of modulation frequencies are used, if there is a difference in the start time of measuring the first frequency and the second frequency, this may cause blurring when measuring the distance to a moving subject.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress blurring of moving subjects.

[0007] A ranging device according to one aspect of the present disclosure includes an emitting unit that emits irradiated light and a light receiving unit that receives reflected light from an object, wherein the irradiated light has a waveform formed by superimposing N modulation frequencies (N is a natural number greater than or equal to 2), and the light receiving unit separates ranging signals obtained from the reflected light for each modulation frequency and outputs distance information that integrates the ranging results obtained from the separated ranging signals.

[0008] A ranging method according to one aspect of the present disclosure includes a ranging device emitting illumination light having a waveform formed by superimposing N (N is a natural number greater than or equal to 2) modulation frequencies, receiving reflected light from an object when the illumination light is reflected from the object, separating ranging signals obtained from the reflected light for each modulation frequency, and outputting distance information that integrates ranging results obtained from the separated ranging signals.

[0009] In one aspect of the ranging device and ranging method of the present disclosure, illumination light having a waveform shape in which N (N is a natural number greater than or equal to 2) modulation frequencies are superimposed is emitted, reflected light from the illumination light reflected by an object is received, ranging signals obtained from the reflected light for each modulation frequency are separated, and distance information that integrates the ranging results obtained from the separated ranging signals is output.

[0010] The distance measuring device according to one aspect of the present disclosure may be an independent device or an internal block constituting a single device.

[0011] 1 is a block diagram showing an example configuration of an embodiment of a distance measuring device to which the present disclosure is applied. FIG. 1 is a block diagram showing a first example configuration of the distance measuring device of FIG. 1. FIG. 2 is a diagram showing an example configuration of a light receiving unit. FIG. 3 is a diagram showing an example planar configuration of a substrate on which a photoelectric conversion unit is formed. FIG. 4 is a diagram showing an example planar configuration of a substrate on which a logic circuit is formed. FIG. 5 is a flowchart explaining the flow of processing performed by the distance measuring device of FIG. 2. FIG. 6 is a diagram showing an example planar structure of a pixel in the first embodiment. FIG. 7 is a diagram showing an example of a light source waveform used in the first embodiment. FIG. 8 is a diagram showing an example of a drive voltage waveform used in the first embodiment. FIG. 9 is a diagram showing an example planar structure of a pixel in the second embodiment. FIG. 10 is a diagram showing an example when the pixel structure of FIG. 10 is compared with the pixel structure of FIG. 7. FIG. 11 is a diagram showing an example circuit configuration of a pixel in the second embodiment. FIG. 12 is a diagram showing an example planar structure of a pixel in the third embodiment. FIG. 13 is a diagram showing an example of a drive voltage waveform used in the third embodiment. FIG. 14 is a diagram showing an example of a pixel structure in the fourth embodiment. FIG. 15 is a diagram showing an example of a drive voltage waveform used in the fourth embodiment. FIG. 16 is a diagram showing an example of a pixel structure in the fifth embodiment. FIG. 17 is a diagram showing an example of a pixel structure in the sixth embodiment. FIG. 18 is a diagram showing an example of a drive voltage waveform used in the sixth embodiment. FIG. 19 is a block diagram showing a second example configuration of the distance measuring device of FIG. 1. 11 is a diagram showing an example of a planar structure of a pixel in a seventh embodiment. FIG. 12 is a diagram showing an example of a drive voltage waveform used in the seventh embodiment. FIG. 13 is a diagram showing an example of a pixel structure in an eighth embodiment. FIG. 14 is a diagram showing an example of a drive voltage waveform used in the eighth embodiment. FIG. 15 is a diagram showing an example of a planar structure of a pixel in a ninth embodiment. FIG. 16 is a diagram showing an example of a drive voltage waveform used in the ninth embodiment. FIG. 17 is a diagram showing an example of a planar structure of a pixel in a tenth embodiment. FIG. 18 is a diagram showing an example of a light source waveform and a drive voltage waveform used in the tenth embodiment. FIG. 19 is a diagram showing an example of a configuration of a polarizer installed on the irradiation surface side of a pixel. FIG. 20 is a diagram showing an example of a planar structure of a pixel in an eleventh embodiment. FIG. 21 is a diagram showing an example of a light source waveform and a drive voltage waveform used in the eleventh embodiment. A block diagram showing an example of a configuration of an embodiment of a distance measuring device to which the present disclosure is applied. FIG. 21 is a diagram showing an example of a configuration of a photodetector element. FIG. 22 is a diagram showing an example of a planar configuration of a substrate on which a photoelectric conversion unit is formed.40. A diagram showing an example of the planar configuration of a substrate on which a logic circuit is formed. A diagram showing an example of the configuration of the pixel light source driving unit of FIG. 35. A diagram showing an example of the configuration of the column processing unit of FIG. 35. A flowchart explaining the flow of processing performed by the distance measuring device of FIG. 32. A diagram showing an example of a superimposed signal of a low-frequency signal and a high-frequency signal. A diagram showing an example of the planar structure of a pixel in the twelfth embodiment. A diagram showing example waveforms of light and driving voltage during distance measurement. A diagram showing example wiring in the pixel array unit of FIG. 35. A diagram showing an example of a driving voltage waveform when distance measurement with a phase shift is performed. A diagram showing an example of a sequence during distance measurement with a phase shift. A diagram showing an example of the circuit configuration of the pixel of FIG. 40. A timing chart including a signal readout operation in the equivalent circuit of the pixel of FIG. 45. A diagram showing an example of a signal output from the readout unit of FIG. 37 to the signal separation unit. A diagram showing an example of processing by the low-frequency signal processing unit and the high-frequency signal processing unit of FIG. 37. A diagram showing an example of processing by the integrated processing unit of FIG. 37. A diagram showing an example of an output format of distance measurement data. A diagram showing an example of volume detection of a moving object. A diagram showing an example of volume detection of a moving object. A diagram showing an example of gesture recognition in a car. 61. A diagram showing an example of the planar configuration of a substrate on which a logic circuit is formed in the thirteenth embodiment. A diagram showing an example of the configuration of the pixel light source drive unit of FIG. 54. A diagram showing an example of a light source waveform used in the thirteenth embodiment. A diagram showing an example of the planar structure of a pixel in the thirteenth embodiment. A diagram showing an example of a drive voltage waveform used in the thirteenth embodiment. A diagram showing an example of wiring in the pixel array unit of FIG. 54. A block diagram showing another example of the configuration of an embodiment of a distance measuring device to which the present disclosure is applied. A diagram showing an example of the planar configuration of a substrate on which a logic circuit is formed in the fourteenth embodiment. A diagram showing an example of the configuration of the pixel light source drive unit of FIG. 61. A diagram showing an example of signals that are switched for use in each distance measuring mode. A flowchart explaining the flow of processing performed by the distance measuring device of FIG. 60. A diagram showing an example of image acquisition of an object by a surveillance camera. A flowchart explaining the flow of processing performed by the distance measuring device of FIG. 60. A diagram showing an example of volume detection of a moving object. A diagram showing an example of volume detection of a moving object.78 is a diagram showing an example of the planar structure of a pixel when distance measurement is performed at one modulation frequency. FIG. 79 is a diagram showing waveforms of reflected light and driving voltage when distance measurement is performed at one modulation frequency. FIG. 80 is a diagram showing waveforms of reflected light and driving voltage when distance measurement is performed at one modulation frequency. FIG. 81 is a diagram showing an example of the planar structure of a pixel when distance measurement is performed by superimposing two modulation frequencies. FIG. 82 is a diagram showing waveforms of reflected light and driving voltage when distance measurement is performed by superimposing two modulation frequencies. FIG. 83 is a diagram showing the relationship between distance measurement accuracy and distance measurement range for each distance measurement method. FIG. 84 is a diagram showing an example configuration of a system including an edge camera and a cloud system equipped with a distance measurement device to which the present disclosure is applied. FIG. 85 is a diagram showing an example configuration of a column processing unit in the sixteenth embodiment. FIG. 86 is a diagram showing an example circuit configuration of a pixel in the sixteenth embodiment. FIG. 87 is a timing chart including a signal readout operation in the equivalent circuit of the pixel of FIG. 77. FIG. 88 is a diagram explaining the effect obtained by a distance measurement device to which the present disclosure is applied. FIG. 89 is a diagram explaining the effect obtained by a distance measurement device to which the present disclosure is applied.

[0012] <Device Configuration> Fig. 1 is a block diagram showing an example configuration of an embodiment of a distance measuring device to which the present disclosure is applied. In Fig. 1, distance measuring device 1 is an indirect ToF distance measuring device. Distance measuring device 1 measures the distance to object 2 by irradiating an object 2 with illumination light IL having a predetermined period and detecting the phase difference between the illumination light IL and reflected light RL. In Fig. 1, distance measuring device 1 is composed of a light emitter 11, a light receiver 12, a distance image output unit 13, and a control unit 14.

[0013] The light-emitting unit 11 has a light source that emits infrared light (IR). The light-emitting unit 11 emits irradiation light IL, which is composed of a light source waveform formed in response to a signal supplied from the light-receiving unit 12, onto the object 2 via a lens or the like. The light-receiving unit 12 has a light-receiving element and a signal processing circuit. The light-receiving unit 12 receives reflected light RL, which is the irradiation light IL reflected by the object 2, via a lens or the like, using the light-receiving element. The light-receiving element is composed of pixels having a photoelectric conversion unit, and generates charges based on the intensity of the received light. The light-receiving unit 12 applies a voltage having a drive voltage waveform corresponding to the light source waveform to the pixels, and performs signal processing on the acquired signal to obtain a distance measurement result.

[0014] The distance image output unit 13 outputs a distance image (distance information) according to the distance measurement result obtained by the light receiving unit 12. The control unit 14 controls the operation of each unit of the distance measuring device 1.

[0015] Fig. 2 is a block diagram showing a first example of the configuration of the distance measuring device 1 of Fig. 1. In Fig. 2, the light emitting unit 11 is made up of a light source 21, a light emission control unit 22, and a modulated signal superimposing unit 23. The light receiving unit 12 is made up of pixels 31 having photoelectric conversion units 31A, a low-frequency signal generating unit 32, a high-frequency signal generating unit 33, a signal separating unit 34, a low-frequency signal processing unit 35, a high-frequency signal processing unit 36, and an integrated processing unit 37.

[0016] The low-frequency signal generation unit 32 generates a low-frequency signal. The high-frequency signal generation unit 33 generates a high-frequency signal. The low-frequency signal and the high-frequency signal are supplied to the modulation signal superimposing unit 23 and the pixel 31. A frequency corresponding to the low-frequency signal and the high-frequency signal is used as a modulation frequency (Fmod). The high-frequency signal is a signal with a higher frequency than the low-frequency signal. Hereinafter, the frequency corresponding to the low-frequency signal is also referred to as a low modulation frequency (L_Fmod), and the frequency corresponding to the high-frequency signal is also referred to as a high modulation frequency (H_Fmod). For example, L_Fmod = 20 MHz, and H_Fmod = 200 MHz can be set.

[0017] The modulation signal superimposing unit 23 superimposes the low-frequency signal supplied from the low-frequency signal generating unit 32 and the high-frequency signal supplied from the high-frequency signal generating unit 33, and supplies the superimposed signal to the light emission control unit 22. Based on the signal supplied from the modulation signal superimposing unit 23, the light emission control unit 22 controls the light source 21 to irradiate the target 2 with illumination light IL having a waveform in which a low modulation frequency and a high modulation frequency are superimposed.

[0018] The pixel 31 receives reflected light RL from the object 2 using the photoelectric conversion unit 31A. The pixel 31 is driven by a voltage having a drive voltage waveform corresponding to the low-frequency signal supplied from the low-frequency signal generation unit 32 and a drive voltage waveform corresponding to the high-frequency signal supplied from the high-frequency signal generation unit 33, thereby obtaining low-frequency and high-frequency signals from the charges generated in the photoelectric conversion unit 31A and supplying the signals to the signal separation unit 34. The signal separation unit 34 separates the signal supplied from the pixel 31 into signals corresponding to the low frequency and the high frequency. The signal separation unit 34 supplies the signal corresponding to the low frequency to a low-frequency signal processing unit 35 and the signal corresponding to the high frequency to a high-frequency signal processing unit 36.

[0019] The low-frequency signal processing unit 35 calculates a ranging result for the low frequency based on the signal corresponding to the low frequency supplied from the signal separation unit 34, and supplies the result to the integration processing unit 37. The high-frequency signal processing unit 36 ​​calculates a ranging result for the high frequency based on the signal corresponding to the high frequency supplied from the signal separation unit 34, and supplies the result to the integration processing unit 37. The integration processing unit 37 integrates the ranging result for the low frequency supplied from the low-frequency signal processing unit 35 and the ranging result for the high frequency supplied from the high-frequency signal processing unit 36, and supplies the resulting distance image (distance information) to the distance image output unit 13.

[0020] The configuration of the light receiving unit 12 will now be described with reference to Figures 3 to 5. Figure 3 is a diagram showing an example of the configuration of the light receiving unit 12. The light receiving unit 12 can be configured as a chip in which a substrate 51 on which a photoelectric conversion unit 31A is formed and a substrate 52 on which a logic circuit is formed are stacked. The inter-substrate connection between the substrates 51 and 52 can be achieved by using a through silicon via (TSV: Through Silicon Via), a Cu-Cu bond in which a direct connection is made using a Cu (copper) terminal formed on the stacking surface, or the like.

[0021] Fig. 4 is a diagram showing an example of the planar configuration of the substrate 51 of Fig. 3. In Fig. 4, a photoelectric conversion unit 31A such as a photodiode (PD) is two-dimensionally formed on the substrate 51. Fig. 5 is a diagram showing an example of the planar configuration of the substrate 52 of Fig. 3. In Fig. 5, the substrate 52 is provided with the low-frequency signal generation unit 32, high-frequency signal generation unit 33, signal separation unit 34, low-frequency signal processing unit 35, high-frequency signal processing unit 36, and integrated processing unit 37 shown in Fig. 2. The substrate 52 is further provided with a vertical control unit 71 that supplies drive signals to the pixel circuits 31B to control them, and the like.

[0022] On the substrate 51 and the substrate 52, the photoelectric conversion unit 31A and the pixel circuit 31B are formed to correspond to each other, and the photoelectric conversion unit 31A and the pixel circuit 31B form pixels 31. The pixels 31 configured in this manner are arranged two-dimensionally to form a pixel array unit 61. Note that the configurations of the light receiving unit 12 shown in Figures 3 to 5 are examples, and other configurations may be adopted.

[0023] <Processing Flow> The processing flow performed by the distance measuring device 1 of FIG. 2 will be described with reference to the flowchart of FIG.

[0024] In step S11, the low-frequency signal generating unit 32 generates a low-frequency signal, and the high-frequency signal generating unit 33 generates a high-frequency signal. In step S12, the modulated signal superimposing unit 23 superimposes the low-frequency signal and the high-frequency signal. In step S13, the light source 21 emits illumination light IL having a modulated waveform toward the object 2.

[0025] In step S14, the photoelectric conversion unit 31A of the pixel 31 receives reflected light RL from the object 2 and simultaneously acquires low-frequency and high-frequency signals. In step S15, the signal separation unit 34 separates the signal acquired by the pixel 31 into signals corresponding to low and high frequencies. In step S16, the low-frequency signal processing unit 35 calculates the distance measurement result for the low frequency, and the high-frequency signal processing unit 36 ​​calculates the distance measurement result for the high frequency.

[0026] In step S17, the integration processing unit 37 integrates the two distance measurement results for the low frequency and the high frequency into one. In step S18, the distance image output unit 13 outputs the distance image obtained by integrating the two distance measurement results.

[0027] In the distance measuring device 1 configured as described above, various configurations can be employed by appropriately changing the waveform shape of the illumination light IL emitted from the light source 21 of the light-emitting unit 11, the structure of the pixels 31 of the light-receiving unit 12, the drive voltage waveform of the voltage applied to the pixels 31, etc. Below, we will explain in order the configurations that can be employed in the distance measuring device 1. In the following explanation, the electrodes or regions used to distribute the charge generated in the photoelectric conversion unit will be referred to as taps.

[0028] <<First embodiment>> In the distance measuring device 1, the light receiving unit 12 includes a pixel array unit 61 in which a plurality of pixels 31 are arranged two-dimensionally. A structure will be described in which a pair of taps is provided in the pixel 31 and two pixels are used as a pair (for example, two adjacent pixels are paired).

[0029] <Pixel Structure> Fig. 7 is a diagram showing an example of the planar structure of a pixel in the first embodiment. Fig. 7 is a diagram of pixel 31 viewed from above the surface of substrate 51. Fig. 7 shows pixel 31-1 and pixel 31-2, which are two pixels arranged adjacent to each other in the column direction (the vertical direction in the figure) in pixel array section 61.

[0030] 7, a photodiode PD is formed as a photoelectric conversion unit 31A in the center of pixel 31-1. A gate electrode 101 of the distribution transistor TG0 and a gate electrode 111 of the distribution transistor TG1 are arranged so as to be line-symmetric (approximately line-symmetric) with respect to a line (hereinafter referred to as the pixel center line) that passes through the center of the photodiode PD and extends in the up-down direction of pixel 31-1. The gate electrodes 101 and 111 are provided so as to overlap at least a portion of the photodiode PD, which is made of an N-type semiconductor region.

[0031] For example, the distribution transistor TG0 includes a gate electrode 101, a gate insulating film formed between the gate electrode 101 and the substrate 51, and an N-type semiconductor region forming a source region and a drain region. In the distribution transistor TG0, the N-type semiconductor region serving as the source region is also used as the photodiode PD, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM0. The distribution transistor TG1 provided with the gate electrode 111 is similar to the distribution transistor TG0 provided with the gate electrode 101. In this way, in the pixel 31-1 of FIG. 7, the gate electrode 101 of the distribution transistor TG0 and the gate electrode 111 of the distribution transistor TG1 are provided as a pair of taps.

[0032] The charge storage unit MEM0 and transfer transistor TRG0, and the charge storage unit MEM1 and transfer transistor TRG1 are arranged symmetrically with respect to the pixel center line and sandwich the photodiode PD and distribution transistors TG0 and TG1 from both sides. For example, the charge storage unit MEM0 comprises an electrode 102, an insulating film provided below the electrode 102, and an N-type semiconductor region provided below the insulating film. In other words, the charge storage unit MEM0 can be formed as a MOS (Metal-Oxide-Semiconductor) capacitor. The transfer transistor TRG0 comprises a gate electrode 103, a gate insulating film formed between the gate electrode 103 and the substrate 51, and N-type semiconductor regions forming the source and drain regions. The charge storage unit MEM1 provided with the electrode 112 and the transfer transistor TRG1 provided with the gate electrode 113 are similar to the charge storage unit MEM0 provided with the electrode 102 and the transfer transistor TRG0 provided with the gate electrode 103.

[0033] A gate electrode 121 of the charge discharging transistor OFG is disposed on the pixel center line (the upper region in the figure). The gate electrode 121 is provided so as to overlap at least a portion of the photodiode PD, which is made up of an N-type semiconductor region. The charge discharging transistor OFG is made up of the gate electrode 121, a gate insulating film formed between the gate electrode 121 and the substrate 51, and N-type semiconductor regions that form the source and drain regions. The charge discharging transistor OFG discharges the charge accumulated in the photodiode PD to the overflow drain OFD.

[0034] In the peripheral region of the photodiode PD (the lower region in the figure), a reset transistor RST, an amplifier transistor AMP, and a select transistor SEL are arranged side by side in the row direction (the left-right direction in the figure). The reset transistor RST comprises a gate electrode 123, a gate insulating film formed between the gate electrode 123 and the substrate 51, and N-type semiconductor regions forming the source and drain regions. In the reset transistor RST, the N-type semiconductor region serving as the source region is also used as the floating diffusion regions FD0 and FD1, and the N-type semiconductor region serving as the drain region is also used as the amplifier transistor AMP.

[0035] The amplifier transistor AMP comprises a gate electrode 124, a gate insulating film formed between the gate electrode 124 and the substrate 51, and an N-type semiconductor region that forms the source region and the drain region. In the amplifier transistor AMP, the N-type semiconductor region that serves as the drain region also serves as the drain region of the reset transistor RST. The select transistor SEL comprises a gate electrode 125, a gate insulating film formed between the gate electrode 125 and the substrate 51, and an N-type semiconductor region that forms the source region and the drain region. In the select transistor SEL, the N-type semiconductor region that serves as the drain region also serves as the source region of the amplifier transistor AMP.

[0036] Pixel 31-2 is configured similarly to pixel 31-1, but for convenience of explanation, the distribution transistors TG0 and TG1 are referred to as distribution transistors TG2 and TG3, and the charge storage units MEM0 and MEM1 are referred to as charge storage units MEM2 and MEM3. Furthermore, in pixel 31-2, the transfer transistors TRG0 and TRG1 of pixel 31-1 are referred to as transfer transistors TRG2 and TRG3, and the floating diffusion regions FD0 and FD1 of pixel 31-1 are referred to as floating diffusion regions FD2 and FD3. That is, in pixel 31-2 of FIG. 7, the gate electrode 101 of the distribution transistor TG2 and the gate electrode 111 of the distribution transistor TG3 are provided as a pair of taps. The planar structures of pixels 31-1 and 31-2 are not limited to the example shown in FIG. 7 and may include, for example, other elements, and are not particularly limited.

[0037] <Light Source Waveform> FIG. 8 is a diagram showing an example of a light source waveform used in the first embodiment. As shown in FIG. 8, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed. For example, L_Fmod = 20 MHz, H_Fmod = 200 MHz, and a waveform formed by multiplying two different Fmod waveforms can be used as the light source waveform. In the example of the light source waveform shown in FIG. 8, the H-level portion of the rectangular wave corresponding to the L_Fmod period is a rectangular wave corresponding to the H_Fmod period.

[0038] <Drive Voltage Waveform> FIG. 9 is a diagram showing an example of a drive voltage waveform used in the first embodiment. In FIG. 9, TG0 represents the drive voltage waveform of the distribution transistor TG0 of the pixel 31-1 of FIG. 7, and TG1 represents the drive voltage waveform of the distribution transistor TG1 of the pixel 31-1 of FIG. 7. The distribution transistors TG0 and TG1 of the pixel 31-1 of FIG. 7 operate with a drive voltage corresponding to the low modulation frequency (L_Fmod). The level of the drive voltage waveform of the distribution transistor TG1 is inverted with respect to the level of the drive voltage waveform of the distribution transistor TG0, and the charge accumulated in the photodiode PD is distributed to the charge accumulation unit MEM0 side and the charge accumulation unit MEM1 side according to the low modulation frequency (L_Fmod).

[0039] 9, TG2 indicates the drive voltage waveform of the distribution transistor TG2 of the pixel 31-2 of FIG. 7, and TG3 indicates the drive voltage waveform of the distribution transistor TG3 of the pixel 31-2 of FIG. 7. The distribution transistors TG2 and TG3 of the pixel 31-2 of FIG. 7 operate with a drive voltage corresponding to the high modulation frequency (H_Fmod). The level of the drive voltage waveform of the distribution transistor TG3 is inverted relative to the level of the drive voltage waveform of the distribution transistor TG2, and the charge accumulated in the photodiode PD is distributed to the charge accumulation unit MEM2 side and the charge accumulation unit MEM3 side according to the high modulation frequency (H_Fmod).

[0040] For example, L_Fmod = 20 MHz, H_Fmod = 200 MHz, and the distribution transistors TG0 and TG1 of pixel 31-1 and the distribution transistors TG2 and TG3 of pixel 31-2 operate using two different Fmods. This makes it possible to calculate distance measurement results with different modulation frequencies (Fmod) for each of pixels 31-1 and 31-2. A distance image is obtained by combining the two distance measurement results obtained in this way into one.

[0041] For example, the distance measurement result for pixel 31-1 at the low modulation frequency (L_Fmod) can be calculated by the following equation (1). Equation (1) represents the I signal on the IQ plane used to measure the distance to the object 2. Q FD0represents the signal based on the charge of the floating diffusion region FD0, and Q FD1 represents the signal based on the charge of the floating diffusion region FD1. FD0 becomes the 0° signal, and Q FD1 is a 180° signal, so the phases are different by 180°.

[0042]

[0043] Furthermore, the distance measurement result for pixel 31-2 at the high modulation frequency (H_Fmod) can be calculated by the following equation (2). Equation (2) represents the I signal, just like equation (1). Q FD2 represents the signal based on the charge of the floating diffusion region FD2, and Q FD3 represents the signal based on the charge of the floating diffusion region FD3. FD2 becomes the 0° signal, and Q FD3 becomes a 180° signal.

[0044]

[0045] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), two pixels arranged in the pixel array unit 61 of the light receiving unit 12 are grouped together, and a pair of taps is provided for each pixel, and voltages with drive voltage waveforms of different modulation frequencies (L_Fmod, H_Fmod) are applied to each pixel. In this pixel structure, the taps are formed of MOS transistors.

[0046] 7 has a structure in which each pixel 31-1 and pixel 31-2 has a photodiode PD, but the two pixels may share the photodiode PD. That is, while Fig. 7 shows a structure in which two pixels 31, each having one pair of taps, are combined, next, a structure in which one pixel 31 is provided with three pairs of taps and charge is distributed in two stages will be described.

[0047] <Pixel Structure> Fig. 10 is a diagram showing an example of the planar structure of a pixel in the second embodiment. As shown in Fig. 10, a photodiode PD is formed as a photoelectric conversion unit 31A in the center of a pixel 31. A gate electrode 201 of the distribution transistor TG0 and a gate electrode 211 of the distribution transistor TG1 are arranged so as to be symmetrical with respect to the pixel center line. The gate electrode 201 and the gate electrode 211 are provided so as to overlap at least a portion of the photodiode PD made of an N-type semiconductor region.

[0048] In FIG. 10, the distribution transistors TG2 and TG3, charge storage units MEM0 and MEM1, transfer transistors TRG0 and TRG1, distribution transistors TG2 and TG3, charge storage units MEM2 and MEM3, and transfer transistors TRG2 and TRG3 are arranged symmetrically with respect to the pixel center line and sandwiching the photodiode PD and distribution transistors TG0 and TG1 from both sides.

[0049] 10 includes a gate electrode 202, a gate insulating film formed between the gate electrode 202 and the substrate 51, and an N-type semiconductor region forming a source region and a drain region. In the distribution transistor TG2, the N-type semiconductor region serving as the source region is also used as the distribution transistor TG0, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM0. The charge storage unit MEM0 provided with an electrode 203 and the transfer transistor TRG0 provided with a gate electrode 204 are the same as the charge storage unit MEM0 and transfer transistor TRG0 in FIG. 7.

[0050] 10 includes a gate electrode 206, a gate insulating film formed between the gate electrode 206 and the substrate 51, and N-type semiconductor regions forming the source and drain regions. In the distribution transistor TG3, the N-type semiconductor region serving as the source region is also used as the distribution transistor TG0, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM1. The charge storage unit MEM1 provided with an electrode 207 and the transfer transistor TRG1 provided with a gate electrode 208 are similar to the charge storage unit MEM2 and transfer transistor TRG2 in FIG. 7.

[0051] 10 includes a gate electrode 212, a gate insulating film formed between the gate electrode 212 and the substrate 51, and an N-type semiconductor region forming a source region and a drain region. In the distribution transistor TG2, the N-type semiconductor region serving as the source region is also used as the distribution transistor TG1, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM2. The charge storage unit MEM2 provided with an electrode 213 and the transfer transistor TRG2 provided with a gate electrode 214 are similar to the charge storage unit MEM1 and transfer transistor TRG1 in FIG. 7.

[0052] 10 includes a gate electrode 216, a gate insulating film formed between the gate electrode 216 and the substrate 51, and N-type semiconductor regions forming the source and drain regions. In this distribution transistor TG3, the N-type semiconductor region serving as the source region is also used as the distribution transistor TG1, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM3. The charge storage unit MEM3 provided with an electrode 217 and the transfer transistor TRG3 provided with a gate electrode 218 are the same as the charge storage unit MEM3 and transfer transistor TRG3 in FIG. 7.

[0053] 10, the charge drain transistor OFG provided with a gate electrode 221 is similar to the charge drain transistor OFG in Fig. 7. The reset transistor RST provided with a gate electrode 223, the amplifier transistor AMP provided with a gate electrode 224, and the select transistor SEL provided with a gate electrode 225 are similar to the reset transistor RST, the amplifier transistor AMP, and the select transistor SEL in Fig. 7. However, in the reset transistor RST, the N-type semiconductor region serving as the source region is also used as the floating diffusion regions FD0, FD1, FD2, and FD3.

[0054] FIG. 11 is a diagram illustrating an example in which the structure of pixel 31 in FIG. 10 is compared with the structures of pixels 31-1 and 31-2 in FIG. 7. As shown in FIG. 11, the pixel 31 in FIG. 10 is configured to read out charge stored in a single photodiode PD by sharing the photodiode PD provided in each of pixels 31-1 and 31-2 in FIG. 7. In pixel 31 in FIG. 10, three pairs of taps are provided: a first pair of taps consisting of the gate electrode 201 of distribution transistor TG0 and the gate electrode 211 of distribution transistor TG1; a second pair of taps consisting of the gate electrode 202 of distribution transistor TG2 and the gate electrode 206 of distribution transistor TG3; and a third pair of taps consisting of the gate electrode 212 of distribution transistor TG2 and the gate electrode 216 of distribution transistor TG3. In pixel 31 in FIG. 10, charge is distributed in two stages using the first pair of taps on the inside and the second or third pair of taps on the outside.

[0055] 12 is a diagram showing an example of the circuit configuration of the pixel 31 in FIG. 10. As shown in FIG. 12, the pixel 31 has a photodiode PD as a photoelectric conversion unit 31A and a charge discharging transistor OFG. One of the source and drain of the charge discharging transistor OFG is electrically connected to the photodiode PD. The other of the source and drain of the charge discharging transistor OFG is electrically connected to a power supply circuit (power supply potential VDD). The charge discharging transistor OFG becomes conductive in response to a voltage applied to its gate, and can discharge the charge accumulated in the photodiode PD to the power supply circuit.

[0056] The pixel 31 also has a distribution transistor TG0, distribution transistors TG2 and TG3, charge storage units MEM0 and MEM1, transfer transistors TRG0 and TRG1, and a distribution transistor TG1, distribution transistors TG2 and TG3, charge storage units MEM2 and MEM3, and transfer transistors TRG2 and TRG3.

[0057] One of the source or drain of the distribution transistor TG0 is electrically connected to the photodiode PD. The other of the source or drain of the distribution transistor TG0 is electrically connected to one of the source or drain of the distribution transistors TG2 and TG3. The other of the source or drain of the distribution transistor TG2 is electrically connected to the charge storage unit MEM0. The other of the source or drain of the distribution transistor TG3 is electrically connected to the charge storage unit MEM1. The distribution transistor TG0 and the distribution transistors TG2 and TG3 become conductive in response to the voltage applied to their gates, and can transfer the charge stored in the photodiode PD to the charge storage units MEM0 and MEM1, respectively.

[0058] One of the source or drain of the transfer transistor TRG0 is electrically connected to the charge storage unit MEM0, and the other of the source or drain is electrically connected to the floating diffusion region FD0. The transfer transistor TRG0 becomes conductive in response to a voltage applied to its gate, and can transfer the charge stored in the charge storage unit MEM0 to the floating diffusion region FD0. One of the source or drain of the transfer transistor TRG1 is electrically connected to the charge storage unit MEM1, and the other of the source or drain is electrically connected to the floating diffusion region FD1. The transfer transistor TRG1 becomes conductive in response to a voltage applied to its gate, and can transfer the charge stored in the charge storage unit MEM1 to the floating diffusion region FD1. The same applies to the distribution transistor TG1, distribution transistors TG2 and TG3, charge storage units MEM2 and MEM3, and transfer transistors TRG2 and TRG3 as to the distribution transistor TG0, distribution transistors TG2 and TG3, charge storage units MEM0 and MEM1, and transfer transistors TRG0 and TRG1.

[0059] The pixel 31 further includes a reset transistor RST, an amplifier transistor AMP, and a select transistor SEL. The floating diffusion regions FD0 and FD1 and the floating diffusion regions FD2 and FD3 are electrically connected to the gate of the amplifier transistor AMP, which converts electric charges into voltages and outputs the converted signals. One of the source and drain of the amplifier transistor AMP is electrically connected to one of the source and drain of the select transistor SEL, which outputs the converted signals to a vertical signal line (VSL) in accordance with a selection signal. The other of the source and drain of the amplifier transistor AMP is electrically connected to a power supply circuit (power supply potential VDD). The other of the source and drain of the select transistor SEL is electrically connected to the vertical signal line (VSL) and a constant current source 226, and is further electrically connected to an AD conversion circuit (not shown). The gate of the select transistor SEL is connected to a drive line that selects a row to output a signal and is electrically connected to the vertical control unit 71 (FIG. 5).

[0060] That is, the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3 are converted into voltages by the amplifier transistor AMP under the control of the select transistor SEL and output to the vertical signal line (VSL). The floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3 are electrically connected to one of the source and drain of a reset transistor RST, which resets the accumulated charges. The gate of the reset transistor RST is electrically connected to a drive line and to the vertical control unit 71 (FIG. 5). The other of the source and drain of the reset transistor RST is electrically connected to a power supply circuit (power supply potential VDD). The reset transistor RST becomes conductive in response to the voltage applied to its gate, and can reset (discharge to the power supply circuit) the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3. Note that the circuit diagram shown in Figure 12 is an example of an equivalent circuit of pixel 31 in Figure 10, and is not limited to the example shown in Figure 12, and may include, for example, other elements, and is not particularly limited.

[0061] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the second embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0062] The drive voltage waveforms used in the second embodiment are similar to those shown in FIG. 9 . That is, in FIG. 9 , TG0 represents the drive voltage waveform of the distribution transistor TG0 of the pixel 31 of FIG. 10 , and TG1 represents the drive voltage waveform of the distribution transistor TG1 of the pixel 31 of FIG. 10 . The distribution transistors TG0 and TG1 of the pixel 31 of FIG. 10 operate with a drive voltage corresponding to the low modulation frequency (L_Fmod). The drive voltage waveforms of the distribution transistors TG0 and TG1 have inverted levels (on / off), and the charge accumulated in the photodiode PD is distributed to the charge storage units MEM0 and MEM1 and the charge storage units MEM2 and MEM3 according to the low modulation frequency (L_Fmod).

[0063] 9, TG2 indicates the drive voltage waveform of the distribution transistor TG2 of the pixel 31 of Fig. 10, and TG3 indicates the drive voltage waveform of the distribution transistor TG3 of the pixel 31 of Fig. 10. The distribution transistors TG2 and TG3 of the pixel 31 of Fig. 10 operate with a drive voltage corresponding to the high modulation frequency (H_Fmod). The drive voltage waveforms of the distribution transistors TG2 and TG3 have inverted levels (on / off), and the charge distributed by the distribution transistors TG0 and TG1 is further distributed according to the high modulation frequency (H_Fmod).

[0064] 10, the charge of the photodiode PD distributed by the distribution transistor TG0 in accordance with the low modulation frequency (L_Fmod) is further distributed to the charge storage unit MEM0 side and the charge storage unit MEM1 side by the distribution transistors TG2 and TG3 in accordance with the high modulation frequency (H_Fmod). Also, in the pixel 31 in FIG. 10, the charge of the photodiode PD distributed by the distribution transistor TG1 in accordance with the low modulation frequency (L_Fmod) is further distributed to the charge storage unit MEM2 side and the charge storage unit MEM3 side by the distribution transistors TG2 and TG3 in accordance with the high modulation frequency (H_Fmod).

[0065] That is, the charge accumulated in the photodiode PD is first distributed by distribution transistors TG0 and TG1, which operate at a low modulation frequency (L_Fmod), and then further distributed by distribution transistors TG2 and TG3, which operate at a high modulation frequency (H_Fmod), thereby distributing the charge in two stages. Here, by changing the combination of FD charges to be added together, it is possible to calculate measurement results using two different modulation frequencies (L_Fmod, H_Fmod).

[0066] For example, the distance measurement result for pixel 31 at the low modulation frequency (L_Fmod) can be calculated by the following equation (3). Equation (3) represents the I signal on the IQ plane used to measure the distance to the object 2. Q FD0represents the signal based on the charge of the floating diffusion region FD0, and Q FD1 represents the signal based on the charge of the floating diffusion region FD1. (Q FD0 + Q FD1 ) is the 0° signal. Q FD2 represents the signal based on the charge of the floating diffusion region FD2, and Q FD3 represents the signal based on the charge of the floating diffusion region FD3. (Q FD2 + Q FD3 ) is a 180° signal.

[0067]

[0068] The distance measurement result for pixel 31 at the high modulation frequency (H_Fmod) can be calculated by the following equation (4). Equation (4) represents the I signal, just like equation (3). (Q FD0 + Q FD2 ) is the 0° signal, and (Q FD1 + Q FD3 ) is a 180° signal. In equations (3) and (4), the distance measurement results for low modulation frequency (L_Fmod) and high modulation frequency (H_Fmod) are calculated by changing the combination of FD charges to be added.

[0069]

[0070] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), three pairs of taps can be provided to each pixel arranged in the pixel array unit 61 of the light receiving unit 12, and voltages with drive voltage waveforms of different modulation frequencies (L_Fmod, H_Fmod) can be applied. By adopting such a configuration, it is possible to reduce the pixel area and improve the resolution.

[0071] In this pixel structure, each tap is formed by a MOS transistor. Here, each pixel is provided with three pairs of taps to distribute charge in two stages. The number of tap pairs (m) formed in one pixel can be expressed by the following equation (5):

[0072] ...(5)

[0073] 10 , the pair of gate electrode 201 of distribution transistor TG0 and gate electrode 211 of distribution transistor TG1, that is, the inner pair of taps among the three pairs of taps, may be configured with vertical gates (VG). Next, a structure in which the inner pair of taps of pixel 31 are configured with VGs will be described.

[0074] <Pixel Structure> Fig. 13 is a diagram showing an example of the planar structure of a pixel in the third embodiment. As shown in Fig. 13, a photodiode PD is formed in the center of a pixel 31, and a gate electrode 301 of a distribution transistor VG0 and a gate electrode 311 of a distribution transistor VG1 are arranged so as to be symmetrical with respect to the pixel center line. For example, the distribution transistors VG0 and VG1 are configured as vertical transistors, with their channels parallel to the substrate 51 and their vertical gates (vertical electrodes) arranged perpendicularly.

[0075] In FIG. 13, the distribution transistors TG0, TG1, charge storage units MEM0, MEM1, transfer transistors TRG0, TRG1, distribution transistors TG0, TG1, charge storage units MEM2, MEM3, and transfer transistors TRG2, TRG3 are arranged symmetrically with respect to the pixel center line and sandwich the photodiode PD and distribution transistors VG0, VG1 from both sides.

[0076] In the left region of Fig. 13, the distribution transistors TG0 and TG1 provided with gate electrodes 302 and 306, the charge storage units MEM0 and MEM1 provided with electrodes 303 and 307, and the transfer transistors TRG0 and TRG1 provided with gate electrodes 304 and 308 are similar to the distribution transistors TG2 and TG3, the charge storage units MEM0 and MEM1, and the transfer transistors TRG0 and TRG1 in the left region of Fig. 10. In the right region of Fig. 13, the distribution transistors TG0 and TG1 provided with gate electrodes 312 and 316, the charge storage units MEM2 and MEM3 provided with electrodes 313 and 317, and the transfer transistors TRG2 and TRG3 provided with gate electrodes 314 and 318 are similar to the distribution transistors TG2 and TG3, the charge storage units MEM2 and MEM3, and the transfer transistors TRG2 and TRG3 in the right region of Fig. 10.

[0077] 13, the charge drain transistor OFG provided with a gate electrode 321, the reset transistor RST provided with a gate electrode 323, the amplifier transistor AMP provided with a gate electrode 324, and the select transistor SEL provided with a gate electrode 325 are the same as the charge drain transistor OFG, reset transistor RST, amplifier transistor AMP, and select transistor SEL in FIG. 10. That is, the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3 are converted into a voltage by the amplifier transistor AMP under the control of the select transistor SEL and output to the vertical signal line (VSL). The reset transistor RST resets the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3.

[0078] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the third embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0079] FIG. 14 is a diagram showing an example of a drive voltage waveform used in the third embodiment. In FIG. 14, VG0 represents the drive voltage waveform of the distribution transistor VG0 of the pixel 31 of FIG. 13, and VG1 represents the drive voltage waveform of the distribution transistor VG1 of the pixel 31 of FIG. 13. The distribution transistors VG0 and VG1 of the pixel 31 of FIG. 13 operate at a drive voltage corresponding to the low modulation frequency (L_Fmod). The drive voltage waveforms of the distribution transistors VG0 and VG1 have inverted levels (on / off), and the charge accumulated in the photodiode PD is distributed to the charge storage units MEM0 and MEM1 and the charge storage units MEM2 and MEM3 according to the low modulation frequency (L_Fmod).

[0080] 14, TG0 denotes the drive voltage waveform of the distribution transistor TG0 of the pixel 31 of FIG. 13, and TG1 denotes the drive voltage waveform of the distribution transistor TG1 of the pixel 31 of FIG. 13. The distribution transistors TG0 and TG1 of the pixel 31 of FIG. 13 operate at a drive voltage corresponding to the high modulation frequency (H_Fmod). The drive voltage waveforms of the distribution transistors TG0 and TG1 have inverted levels (on / off), and the charge distributed by the distribution transistors VG0 and VG1 is further distributed according to the high modulation frequency (H_Fmod). That is, the charge accumulated in the photodiode PD is distributed by the distribution transistors VG0 and VG1 operating at the low modulation frequency (L_Fmod), and then further distributed by the distribution transistors TG0 and TG1 operating at the high modulation frequency (H_Fmod), thereby distributing the charge in two stages.

[0081] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), it is possible to configure the distance measuring device 1 so that one inner pair of taps out of three pairs of taps provided for each pixel arranged in the pixel array unit 61 of the light receiving unit 12 is made VG and a voltage having a drive voltage waveform with a different modulation frequency (L_Fmod, H_Fmod) is applied. By adopting such a configuration, the inner pair of VG taps distributes charge to the left and right (left and right directions in FIG. 13 ), thereby strengthening the electric field deep in the photodiode PD and improving the charge transfer efficiency (Cmod).

[0082] 10 shows an example of the pixel 31 having a gate structure, but it may have a CAPD (Current Assisted Photonic Demodulator) structure. Next, a case where the pixel structure of the pixel 31 is the CAPD structure will be described.

[0083] <Pixel Structure> Fig. 15 is a diagram showing an example of a pixel structure in the fourth embodiment. Fig. 15 shows the planar structure of pixel 31, the structure of the A-A' cross section of the planar structure, and the structure of the B-B' cross section of the planar structure. As shown in the planar structure of Fig. 15, pixel 31 has an N-type semiconductor region 411 as a photoelectric conversion unit, and a P-type semiconductor region 412 which is a vertically elongated region and a P-type semiconductor region 413 which is a circular region are formed so as to be line-symmetrical with respect to the pixel center line. An N-type semiconductor region 414 is formed so as to surround the periphery of P-type semiconductor region 413. The N-type semiconductor region 411 and the N-type semiconductor region 414 have different impurity concentrations.

[0084] In the planar structure of Figure 15, pixel 31 has a structure in which semiconductor regions P2 and P3, each made of a circular P-type semiconductor region 413 surrounded by an N-type semiconductor region 414, are formed to sandwich two vertically elongated semiconductor regions P0 and P1 made of a P-type semiconductor region 412. Two of each of the semiconductor regions P2 and P3 are formed symmetrically about the pixel center line, for a total of four. Of the two semiconductor regions P2, one semiconductor region P2 is surrounded by a semiconductor region N0, and the other semiconductor region P2 is surrounded by a semiconductor region N2. Of the two semiconductor regions P3, one semiconductor region P3 is surrounded by a semiconductor region N1, and the other semiconductor region P3 is surrounded by a semiconductor region N3.

[0085] Voltage application lines for applying voltages are electrically connected to the semiconductor regions P0, P1, P2, and P3, and voltages can be applied through the voltage application lines. Signal output lines for extracting charges are electrically connected to the semiconductor regions N0, N1, N2, and N3. Here, the signal output section is composed of the semiconductor region P2 and the semiconductor region N0. In other words, the signal output section has the semiconductor region P2 as the voltage application section located in the center and the semiconductor region N0 as the charge detection section located around it. The charges detected in the semiconductor region N0 are transferred to the floating diffusion region FD0 via the signal output lines and stored therein.

[0086] The same applies to the semiconductor region P3 and semiconductor region N1, the semiconductor region P2 and semiconductor region N2, and the semiconductor region P3 and semiconductor region N3. In the pixel 31 in Fig. 15 , the charges transferred from the semiconductor regions N0, N1, N2, and N3 and accumulated in the floating diffusion regions FD0, FD1, FD2, and FD3 are converted into a voltage by the amplification transistor AMP under the control of the selection transistor SEL and output to the vertical signal line (VSL).

[0087] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the fourth embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0088] FIG. 16 is a diagram showing an example of a drive voltage waveform used in the fourth embodiment. In FIG. 16, P0 represents the drive voltage waveform of the semiconductor region P0 of the pixel 31 in FIG. 15, and P1 represents the drive voltage waveform of the semiconductor region P1 of the pixel 31 in FIG. 15. The semiconductor regions P0 and P1 of the pixel 31 in FIG. 15 operate with a drive voltage corresponding to a low modulation frequency (L_Fmod). The drive voltage waveforms of the semiconductor region P0 and the semiconductor region P1 have inverted levels (on / off), and the charge generated in the N-type semiconductor region 411 is distributed between the semiconductor region P0 side and the semiconductor region P1 side according to the low modulation frequency (L_Fmod).

[0089] 16, P2 indicates the drive voltage waveform of the semiconductor region P2 of the pixel 31 in FIG. 15, and P3 indicates the drive voltage waveform of the semiconductor region P3 of the pixel 31 in FIG. 15. The semiconductor regions P2 and P3 of the pixel 31 in FIG. 15 operate with a drive voltage corresponding to a high modulation frequency (H_Fmod). The drive voltage waveforms of the semiconductor regions P2 and P3 have inverted levels (on / off), and the charge distributed to the semiconductor region P0 side or the semiconductor region P1 side is distributed to the semiconductor region P2 side and the semiconductor region P3 side in accordance with the high modulation frequency (H_Fmod).

[0090] For example, as shown in FIG. 16 , the movement of electrons e when the drive voltage waveforms of the semiconductor regions at time t1 are at the H level (on state), the drive voltage waveform of the semiconductor region P0 is at the L level (off state), the drive voltage waveform of the semiconductor region P1 is at the H level (on state), and the drive voltage waveform of the semiconductor region P3 is at the L level (off state) is shown in FIG. 15 . That is, as shown in the A-A′ cross section of FIG. 15 , when the drive voltage waveform (L_Fmod) of the semiconductor region P0 is in the ON state, electrons e move toward the semiconductor region P0. Furthermore, as shown in the B-B′ cross section of FIG. 15 , when the drive voltage waveform (H_Fmod) of the semiconductor region P2 is in the ON state, electrons e move toward the semiconductor region P2. As a result, as shown by the arrows in the planar structure of FIG. 15 , electrons e are first sorted toward the semiconductor region P0, and then further sorted toward the semiconductor region P2.

[0091] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light-emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), three pairs of taps can be provided for each pixel arranged in the pixel array unit 61 of the light-receiving unit 12, and voltages with drive voltage waveforms of the different modulation frequencies (L_Fmod, H_Fmod) can be applied. In this pixel structure, the taps are formed of impurities. Here, each pixel is provided with three pairs of taps to distribute charge in two stages, and the number of tap pairs (m) formed within one pixel can be expressed by the following equation (5):

[0092] ...(5)

[0093] By adopting such a configuration, pixels can be formed in the impurity region without forming gate electrodes, thereby reducing manufacturing costs.

[0094] 15 shows a pixel 31 having a CAPD structure, but the layout of the semiconductor region may be changed. Next, a modified example of the layout when the pixel 31 has a CAPD structure will be described.

[0095] <Pixel Structure> Fig. 17 is a diagram showing an example of a pixel structure in the fifth embodiment. Similar to Fig. 15, Fig. 17 shows the planar structure of pixel 31, the A-A' cross-sectional structure of the planar structure, and the B-B' cross-sectional structure of the planar structure. As shown in the planar structure of Fig. 17, pixel 31 has an N-type semiconductor region 411 as a photoelectric conversion unit, and a P-type semiconductor region 413 which is a circular region and a P-type semiconductor region 412 which is a vertically elongated region are formed so as to be line-symmetrical with respect to the pixel center line. An N-type semiconductor region 414 is formed so as to surround the periphery of P-type semiconductor region 413.

[0096] In the planar structure of FIG. 17 , pixel 31 has a structure in which vertically elongated semiconductor regions P0 and P1 made of P-type semiconductor regions 412 are formed to sandwich four semiconductor regions P2 and P3 made of circular P-type semiconductor regions 413 on both sides. Two semiconductor regions P2 and P3 are formed on each side so as to be symmetrical with respect to the pixel center line, for a total of four semiconductor regions. The four semiconductor regions P2 and P3 and the semiconductor regions N0, N1, N2, and N3 surrounding them are similar to the semiconductor regions P2 and P3 and semiconductor regions N0, N1, N2, and N3 in FIG. 15 . Furthermore, the vertically elongated semiconductor regions P0 and P1 are similar to the semiconductor regions P0 and P1 in FIG. 15 .

[0097] Comparing the layout of pixel 31 in Fig. 17 with the layout of pixel 31 in Fig. 15, the following can be seen: In pixel 31 in Fig. 15, vertically elongated semiconductor regions P0 and P1 are arranged inside four circular semiconductor regions P2 and P3, but in pixel 31 in Fig. 17, vertically elongated semiconductor regions P0 and P1 are arranged outside four circular semiconductor regions P2 and P3.

[0098] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the fifth embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0099] The drive voltage waveforms used in the fifth embodiment are similar to those shown in Fig. 16. That is, in Fig. 16, P0 indicates the drive voltage waveform of the semiconductor region P0 of the pixel 31 in Fig. 17, and P1 indicates the drive voltage waveform of the semiconductor region P1 of the pixel 31 in Fig. 17. Also in Fig. 16, P2 indicates the drive voltage waveform of the semiconductor region P2 of the pixel 31 in Fig. 17, and P3 indicates the drive voltage waveform of the semiconductor region P3 of the pixel 31 in Fig. 17.

[0100] For example, as shown in FIG. 16 , at time t1, the drive voltage waveform for semiconductor region P0 is at H level (on state), the drive voltage waveform for semiconductor region P1 is at L level (off state), the drive voltage waveform for semiconductor region P2 is at H level (on state), and the drive voltage waveform for semiconductor region P3 is at L level (off state). That is, as shown in the A-A′ cross section of FIG. 17 , the drive voltage waveform (L_Fmod) for semiconductor region P0 is in the on state, causing electrons e to move toward semiconductor region P0. Also, as shown in the B-B′ cross section of FIG. 17 , the drive voltage waveform (H_Fmod) for semiconductor region P2 is in the on state, causing electrons e to move toward semiconductor region P2. As a result, as shown by the arrows in the planar structure of FIG. 17 , electrons e are first sorted toward semiconductor region P0, and then further sorted toward semiconductor region P2.

[0101] As described above, in the distance measuring device 1, of the three pairs of taps formed with impurities in each pixel arranged in the pixel array section 61 of the light receiving unit 12, one pair of taps to which a voltage with a drive voltage waveform corresponding to the low modulation frequency (L_Fmod) is applied can be configured inside or outside the other two pairs of taps to which a voltage with a drive voltage waveform corresponding to the high modulation frequency (H_Fmod) is applied. By adopting such a configuration, pixels can be formed using only impurities, thereby reducing manufacturing costs.

[0102] 10 shows an example of the pixel 31 having a gate structure, but it may have a CDTI (Capacitive Deep Trench Isolation) structure. Next, a case where the pixel structure of the pixel 31 is the CDTI structure will be described.

[0103] <Pixel Structure> FIG. 18 is a diagram illustrating an example of a pixel structure in the sixth embodiment. FIG. 18 illustrates the planar structure of a pixel 31 and the structure of an A-A' cross section of the planar structure. As shown in the planar structure of FIG. 18, the pixel 31 has an N-type semiconductor region 451 as a photoelectric conversion unit, and an insulator 453 is formed to surround the periphery of the N-type semiconductor region 451. L-shaped voltage application units CDTI-A, CDTI-B, CDTI-C, and CDTI-D are respectively disposed at the four corners of the insulator 453. As shown in the A-A' cross section of FIG. 18, the voltage application unit CDTI-A is formed by forming a groove in a substrate, covering the groove with an insulator 453, and embedding a conductor 454. A voltage application line for applying a voltage is electrically connected to the voltage application unit CDTI-A, and a voltage can be applied through the voltage application line. The voltage application units CDTI-B, CDTI-C, and CDTI-D are similar to the voltage application unit CDTI-A. As shown in the AA' cross section of Figure 18, voltage application units CDTI-A, CDTI-B, CDTI-C, and CDTI-D are vertical transfer CDTIs that transfer electrons e in the vertical direction in response to a voltage applied through a voltage application line.

[0104] As shown in the planar structure of FIG. 18 , the voltage application units CDTI-1 and CDTI-3 are arranged opposite each other so as to be line-symmetric (approximately line-symmetric) with respect to a line that passes through the center of the N-type semiconductor region 451 and extends along the vertical direction of the pixel 31. Furthermore, the voltage application units CDTI-0 and CDTI-2 are arranged opposite each other so as to be line-symmetric (approximately line-symmetric) with respect to a line that passes through the center of the N-type semiconductor region 451 and extends along the horizontal direction of the pixel 31. The voltage application unit CDTI-0, like the voltage application unit CDTI-A, is formed by forming a groove in the substrate, covering the groove with an insulator 453, and burying a conductor 454. A voltage application line for applying a voltage is electrically connected to the voltage application unit CDTI-0, and a voltage can be applied via the voltage application line. The voltage application units CDTI-1, CDTI-2, and CDTI-3 are similar to the voltage application unit CDTI-0. As shown in the planar structure and AA' cross section of Figure 18, voltage application units CDTI-0, CDTI-1, CDTI-2, and CDTI-3 are horizontal transfer CDTIs that transfer electrons e in the horizontal direction in response to voltages applied through voltage application lines.

[0105] As shown in the planar structure of FIG. 18 , rectangular charge detection units FD-A, FD-B, FD-C, and FD-D are arranged at the four corners of a diagonal line passing through the center of an N-type semiconductor region 451. As shown in the A-A' cross section of FIG. 18 , the charge detection unit FD-A is formed from an N-type semiconductor region 452. The N-type semiconductor region 452 is an N+ impurity region with a different impurity concentration than the N-type semiconductor region 451, which is an N- impurity region. A signal output line for extracting charge is electrically connected to the charge detection unit FD-A. The charge detected by the charge detection unit FD-A is transferred to and accumulated in the floating diffusion region FD0 via the signal output line. The charge detection units FD-B, FD-C, and FD-D are similar to the charge detection unit FD-A. In the pixel 31 of FIG. 18 , the charges transferred from the charge detection units FD-A, FD-B, FD-C, and FD-D and accumulated in the floating diffusion regions FD0, FD1, FD2, and FD3 are converted into a voltage by the amplification transistor AMP under the control of the selection transistor SEL, and are output to the vertical signal line (VSL).

[0106] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the sixth embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0107] FIG. 19 is a diagram showing an example of a drive voltage waveform used in the sixth embodiment. In FIG. 19, CDTI-1 indicates the drive voltage waveform of the voltage application unit CDTI-1 of the pixel 31 of FIG. 18, and CDTI-3 indicates the drive voltage waveform of the voltage application unit CDTI-3 of the pixel 31 of FIG. 18. The voltage application units CDTI-1 and CDTI-3, which are arranged opposite each other in the pixel 31 of FIG. 18, operate at a drive voltage corresponding to a low modulation frequency (L_Fmod). The drive voltage waveforms of the voltage application units CDTI-1 and CDTI-3 have inverted levels (on / off), and charge generated in the N-type semiconductor region 451 is distributed between the voltage application unit CDTI-1 side and the voltage application unit CDTI-3 side according to the low modulation frequency (L_Fmod).

[0108] 19, CDTI-0 indicates the drive voltage waveform of the voltage application unit CDTI-0 of the pixel 31 in Fig. 18, and CDTI-2 indicates the drive voltage waveform of the voltage application unit CDTI-2 of the pixel 31 in Fig. 18. The voltage application units CDTI-0 and CDTI-2, which are arranged opposite each other in the pixel 31 in Fig. 18, operate at a drive voltage corresponding to the high modulation frequency (H_Fmod). The drive voltage waveforms of the voltage application units CDTI-0 and CDTI-2 have inverted levels (on / off), and the charge generated in the N-type semiconductor region 451 is distributed between the voltage application unit CDTI-0 side and the voltage application unit CDTI-2 side according to the high modulation frequency (H_Fmod).

[0109] For example, in FIG. 19, the movement of electrons e at the timing when the drive voltage waveform of the voltage application unit CDTI-1 is at the H level (on state), the drive voltage waveform of the voltage application unit CDTI-3 is at the L level (off state), the drive voltage waveform of the voltage application unit CDTI-0 is at the H level (on state), and the drive voltage waveform of the voltage application unit CDTI-2 is at the L level (off state) is shown in FIG. 18. That is, as shown in the planar structure of FIG. 18, when the voltage application units CDTI-1 and CDTI-0 are turned on, electrons e move toward the charge detection unit FD-A (moving in the direction of the arrow in the figure). Furthermore, as shown in the A-A' cross section of pixel 31, because the voltage application units CDTI-1 and CDTI-0 are CDTIs for horizontal transfer, when the voltage application unit CDTI-A for vertical transfer is turned on, electrons e move toward the charge detection unit FD-A. During charge transfer, the vertical transfer voltage application units CDTI-A to CDTI-D may be kept in an ON state at all times.

[0110] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), a CDTI structure can be adopted as the structure of each pixel arranged in the pixel array unit 61 of the light receiving unit 12, and a voltage with a drive voltage waveform of a different modulation frequency (L_Fmod, H_Fmod) can be applied to each of the horizontal transfer CDTIs arranged opposite to each other. By adopting such a configuration and transferring charges using the CDTIs, there is an advantage that the charge transfer efficiency (Cmod) is less likely to decrease even during high-speed operation. Note that when N modulation frequencies (N is a natural number greater than or equal to 2) are used, each tap (horizontal transfer CDTI) has two taps per pixel. N It can also be said that individuals are formed.

[0111] <<Seventh Embodiment>> In the pixel 31 shown in Fig. 10, three pairs of taps are provided in one pixel 31, but four taps may be provided in one pixel 31. Next, a structure in which four taps are provided in one pixel 31 will be described. When a four-tap pixel structure is adopted, a waveform generated by superimposing (multiplying) two different modulation frequencies (Fmod) is used as the drive voltage waveform, so the configuration of the distance measuring device 1 in Fig. 1 can be changed to the configuration shown in Fig. 20.

[0112] <Device Configuration> Figure 20 is a block diagram showing a second example of the configuration of the distance measuring device of Figure 1. In Figure 20, parts corresponding to those in Figure 2 are assigned the same reference numerals, and their description will be omitted as appropriate. Compared to the distance measuring device 1 of Figure 2, the distance measuring device 1 of Figure 20 is provided with a light receiving unit 12A instead of the light receiving unit 12. Compared to the light receiving unit 12, the light receiving unit 12A is additionally provided with a modulation signal superimposing unit 38. The modulation signal superimposing unit 38 superimposes the low-frequency signal supplied from the low-frequency signal generating unit 32 and the high-frequency signal supplied from the high-frequency signal generating unit 33, and supplies the superimposed signal to the pixel 31.

[0113] The pixel 31 is driven by a voltage having a drive voltage waveform corresponding to the superimposed signal supplied from the modulation signal superimposing unit 38, thereby obtaining low-frequency and high-frequency signals from the charges generated in the photoelectric conversion unit 31A and supplying them to the signal separating unit 34. When the distance measuring device 1 employs the configuration shown in Fig. 20 and the light receiving unit 12A is configured as a chip in which the substrate 51 and the substrate 52 are stacked, a modulation signal superimposing unit 38 is provided on the substrate 52 (Fig. 5) for the low-frequency signal generating unit 32 and the high-frequency signal generating unit 33, although this is not shown, and the superimposed signal from the modulation signal superimposing unit 38 is input to the pixel 31 (pixel circuit 31B).

[0114] <Pixel Structure> Fig. 21 is a diagram showing an example of the planar structure of a pixel in the seventh embodiment. As shown in Fig. 21, a photodiode PD is formed in the center of pixel 31, and a gate electrode 501 of distribution transistor TG0, a gate electrode 511 of distribution transistor TG1, a gate electrode 521 of distribution transistor TG2, and a gate electrode 531 of distribution transistor TG3 are arranged so as to be line-symmetrical with respect to the pixel center line. The gate electrodes 501, 511, 521, and 531 are provided so as to overlap at least a portion of the photodiode PD made of an N-type semiconductor region.

[0115] The distribution transistor TG0 comprises a gate electrode 501, a gate insulating film formed between the gate electrode 501 and the substrate 51, and an N-type semiconductor region forming a source region and a drain region. In the distribution transistor TG0, the N-type semiconductor region serving as the source region is also used as the photodiode PD, and the N-type semiconductor region serving as the drain region is also used as the charge storage unit MEM0. The distribution transistors TG1, TG2, and TG3 are similar to the distribution transistor TG0.

[0116] In FIG. 21, charge accumulation units MEM0 and MEM1, transfer transistors TRG0 and TRG1, charge accumulation units MEM2 and MEM3, and transfer transistors TRG2 and TRG3 are arranged symmetrically with respect to the pixel center line and sandwiching the photodiode PD, distribution transistors TG0 and TG1, and distribution transistors TG2 and TG3 from both sides.

[0117] In the left region of Fig. 21, the charge storage units MEM0 and MEM1 provided with electrodes 502 and 512 and the transfer transistors TRG0 and TRG1 provided with gate electrodes 503 and 513 are similar to the charge storage units MEM0 and MEM1 and the transfer transistors TRG0 and TRG1 in the left region of Fig. 10. In addition, in the right region of Fig. 21, the charge storage units MEM2 and MEM3 provided with electrodes 522 and 532 and the transfer transistors TRG2 and TRG3 provided with gate electrodes 523 and 533 are similar to the charge storage units MEM2 and MEM3 and the transfer transistors TRG2 and TRG3 in the right region of Fig. 10.

[0118] 21, the charge drain transistor OFG provided with a gate electrode 541, the reset transistor RST provided with a gate electrode 543, the amplifier transistor AMP provided with a gate electrode 544, and the select transistor SEL provided with a gate electrode 545 are the same as the charge drain transistor OFG, reset transistor RST, amplifier transistor AMP, and select transistor SEL in FIG. 10. That is, the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3 are converted into a voltage by the amplifier transistor AMP under the control of the select transistor SEL and output to the vertical signal line (VSL). The reset transistor RST resets the charges accumulated in the floating diffusion regions FD0, FD1 and the floating diffusion regions FD2, FD3.

[0119] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the seventh embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0120] Fig. 22 is a diagram showing an example of a drive voltage waveform used in the seventh embodiment. In Fig. 22, TG0 indicates the drive voltage waveform of the distribution transistor TG0 of the pixel 31 of Fig. 21, and TG1 indicates the drive voltage waveform of the distribution transistor TG1 of the pixel 31 of Fig. 21. Also in Fig. 22, TG2 indicates the drive voltage waveform of the distribution transistor TG2 of the pixel 31 of Fig. 21, and TG3 indicates the drive voltage waveform of the distribution transistor TG3 of the pixel 31 of Fig. 21.

[0121] 22, the distribution transistors TG0, TG1 and the distribution transistors TG2, TG3 are alternately driven for each cycle of the low modulation frequency (L_Fmod), and after the distribution transistors TG0, TG1 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod), the distribution transistors TG2, TG3 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod). That is, by applying voltages to the gate electrodes of the distribution transistors TG0, TG1, TG2, TG3 at the timing of the drive voltage waveforms in FIG. 22, the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD0, FD1, FD2, FD3.

[0122] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), four taps can be provided to each pixel arranged in the pixel array unit 61 of the light receiving unit 12, and a drive voltage waveform voltage having a waveform generated by multiplying the two different modulation frequencies (L_Fmod, H_Fmod) can be applied. In this pixel structure, each tap is formed by a MOS transistor. By adopting such a configuration, the number of taps per pixel becomes four, which is advantageous for miniaturizing the pixels. Note that when N modulation frequencies (N is a natural number equal to or greater than 2) are used, each tap is provided with two taps per pixel. N It can also be said that individuals are formed.

[0123] <<Eighth Embodiment>> Although the pixel 31 shown in Fig. 21 has a gate structure when a four-tap pixel structure is adopted, the pixel 31 may have a CAPD structure. Next, a case where the pixel 31 has a CAPD structure when a four-tap pixel structure is adopted will be described. However, when a four-tap pixel structure is adopted, a waveform generated by superimposing (multiplying) two different modulation frequencies (Fmod) is used, and therefore the configuration of the distance measuring device 1 becomes the configuration shown in Fig. 20.

[0124] <Pixel Structure> Fig. 23 is a diagram showing an example of a pixel structure in the eighth embodiment. Fig. 23 shows the planar structure of pixel 31 and the structure of the A-A' cross section of the planar structure. As shown in the planar structure of Fig. 23, pixel 31 has an N-type semiconductor region 411 as a photoelectric conversion unit, and P-type semiconductor regions 413, which are circular regions, are formed in four locations so as to be line-symmetrical with respect to the pixel center line. N-type semiconductor regions 414 are formed so as to surround the periphery of P-type semiconductor region 413.

[0125] 23, pixel 31 has a structure in which four semiconductor regions P0, P1, P2, and P3 each made of a circular P-type semiconductor region 413 are formed, and around them are formed semiconductor regions N0, N1, N2, and N3 each made of an N-type semiconductor region 414. That is, semiconductor region P0 surrounded by semiconductor region N0, semiconductor region P1 surrounded by semiconductor region N1, semiconductor region P2 surrounded by semiconductor region N2, and semiconductor region P3 surrounded by semiconductor region N3 are formed so as to be symmetrical with respect to the pixel center line.

[0126] Voltage application lines are electrically connected to the semiconductor regions P0, P1, P2, and P3, and voltage can be applied through the voltage application lines. Signal output lines are electrically connected to the semiconductor regions N0, N1, N2, and N3, and signal output lines are used to extract charge. Here, the signal output section is made up of the semiconductor region P0 as the voltage application section and the semiconductor region N0 as the charge detection section, and the charge detected in the semiconductor region N0 is transferred to and stored in the floating diffusion region FD0 through the signal output lines.

[0127] The same applies to semiconductor region P0 and semiconductor region N0 as to semiconductor region P1 and semiconductor region N1, semiconductor region P2 and semiconductor region N2, and semiconductor region P3 and semiconductor region N3. In pixel 31 of Fig. 23 , the charges transferred from semiconductor regions N0, N1, N2, and N3 and accumulated in floating diffusion regions FD0, FD1, FD2, and FD3 are converted into a voltage by the amplification transistor AMP under the control of the selection transistor SEL and output to the vertical signal line (VSL).

[0128] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the eighth embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0129] Fig. 24 is a diagram showing an example of a drive voltage waveform used in the eighth embodiment. In Fig. 24, P0 indicates the drive voltage waveform of the semiconductor region P0 of the pixel 31 in Fig. 23, and P1 indicates the drive voltage waveform of the semiconductor region P1 of the pixel 31 in Fig. 23. Also in Fig. 24, P2 indicates the drive voltage waveform of the semiconductor region P2 of the pixel 31 in Fig. 23, and P3 indicates the drive voltage waveform of the semiconductor region P3 of the pixel 31 in Fig. 23.

[0130] 24, the semiconductor regions P0, P1 and the semiconductor regions P2, P3 are alternately driven for each cycle of the low modulation frequency (L_Fmod), and after the semiconductor regions P0, P1 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod), the semiconductor regions P2, P3 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod). That is, by applying voltage to the semiconductor regions P0, P1, P2, P3 through voltage application lines at the timing of the drive voltage waveform in FIG. 24, charge generated in the N-type semiconductor region 411 serving as a photoelectric conversion unit is transferred to one of the floating diffusion regions FD0, FD1, FD2, FD3.

[0131] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), four taps can be provided to each pixel arranged in the pixel array unit 61 of the light receiving unit 12, and a drive voltage waveform voltage having a waveform generated by multiplying the two different modulation frequencies (L_Fmod, H_Fmod) can be applied. In this pixel structure, each tap is formed of impurities. By adopting such a configuration, the number of taps per pixel becomes four, which is advantageous for miniaturizing pixels. Furthermore, since pixels can be formed only with impurities, manufacturing costs can be reduced. Note that when N modulation frequencies (N is a natural number greater than or equal to 2) are used, each tap is formed of 2 taps per pixel. N It can also be said that individuals are formed.

[0132] <<Ninth embodiment>> When the four-tap pixel structure shown in Fig. 21 is adopted, the pixel 31 may have a pixel structure using a photogate (PG: Photo-Gate). Next, a case where the pixel 31 has a structure using a photogate will be described. However, when the four-tap pixel structure is adopted, a waveform generated by superimposing (multiplying) two different modulation frequencies (Fmod) is used, and therefore the configuration of the distance measuring device 1 becomes the configuration shown in Fig. 20.

[0133] <Pixel Structure> Fig. 25 is a diagram showing an example of the planar structure of a pixel in the ninth embodiment. As shown in Fig. 25, in pixel 31, a gate electrode 601 of photogate PG0, a gate electrode 611 of photogate PG1, a gate electrode 621 of photogate PG2, and a gate electrode 631 of photogate PG3 are arranged so as to have a symmetrical structure with respect to the center portion. Photodiodes PD are formed below photogates PG1 to PG4.

[0134] The photogate PG0 is provided with a charge storage unit MEM0 provided with an electrode 602 and a transfer transistor TRG0 provided with a gate electrode 603. The photogate PG1 is provided with a charge storage unit MEM1 provided with an electrode 612 and a transfer transistor TRG1 provided with a gate electrode 613. The photogate PG2 is provided with a charge storage unit MEM2 provided with an electrode 622 and a transfer transistor TRG2 provided with a gate electrode 623. The photogate PG3 is provided with a charge storage unit MEM3 provided with an electrode 632 and a transfer transistor TRG3 provided with a gate electrode 633.

[0135] 25, the reset transistor RST provided with a gate electrode 641, the amplifier transistor AMP provided with a gate electrode 642, and the select transistor SEL provided with a gate electrode 643 are the same as the reset transistor RST, the amplifier transistor AMP, and the select transistor SEL in FIG. 21. That is, the charges accumulated in the floating diffusion regions FD0, FD1, FD2, and FD3 are converted into a voltage by the amplifier transistor AMP under the control of the select transistor SEL and output to the vertical signal line (VSL). The reset transistor RST resets the charges accumulated in the floating diffusion regions FD0, FD1, FD2, and FD3.

[0136] <Light Source Waveform / Driving Voltage Waveform> The light source waveform used in the ninth embodiment is the same as the light source waveform shown in Fig. 8. That is, the illumination light IL emitted from the light source 21 has a waveform shape in which a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed.

[0137] Fig. 26 is a diagram showing an example of a drive voltage waveform used in the ninth embodiment. In Fig. 26, PG0 indicates the drive voltage waveform of the photogate PG0 of the pixel 31 of Fig. 25, and PG1 indicates the drive voltage waveform of the photogate PG1 of the pixel 31 of Fig. 25. Also in Fig. 26, PG2 indicates the drive voltage waveform of the photogate PG2 of the pixel 31 of Fig. 25, and PG3 indicates the drive voltage waveform of the photogate PG3 of the pixel 31 of Fig. 25.

[0138] 26, the photogates PG0 and PG1 and the photogates PG2 and PG3 are alternately driven for each cycle of the low modulation frequency (L_Fmod), and after the photogates PG0 and PG1 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod), the photogates PG2 and PG3 alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod). That is, by applying voltage to the gate electrodes of the photogates PG0, PG1, PG2, and PG3 at the timing of the drive voltage waveform in FIG. 26, the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD0, FD1, FD2, and FD3.

[0139] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light-emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), four taps can be provided to each pixel arranged in the pixel array unit 61 of the light-receiving unit 12, and a drive voltage waveform having a waveform generated by multiplying the two different modulation frequencies (L_Fmod, H_Fmod) can be applied. In this pixel structure, each tap is formed by a photogate. By adopting such a configuration, the number of taps per pixel becomes four, which is advantageous for miniaturizing the pixels. Note that in the pixel 31 of FIG. 25, photogates PG1 to PG4 may be configured to generate charge.

[0140] <<Tenth Embodiment>> In the configurations shown in Figures 20 to 22, two different modulation frequencies (Fmod) are superimposed, but the number of superimposed modulation frequencies (Fmod) may be three or more. Next, a configuration in which three different modulation frequencies (Fmod) are superimposed will be described. The distance measuring device 1 has the configuration shown in Figure 20, but instead of the low-frequency signal generator 32 and the high-frequency signal generator 33, a generator (not shown) is provided that generates frequency signals corresponding to three modulation frequencies (Fmod1, Fmod2, Fmod3). Furthermore, the modulation signal superimposing units 23 and 38 superimpose the three modulation frequencies (Fmod1, Fmod2, Fmod3).

[0141] 27 is a diagram showing an example of the planar structure of a pixel in the tenth embodiment. As shown in Fig. 27, a photodiode PD is formed in the center of pixel 31, and a gate electrode 701 of distributing transistor TG0, a gate electrode 711 of distributing transistor TG1, a gate electrode 721 of distributing transistor TG2, a gate electrode 731 of distributing transistor TG3, a gate electrode 741 of distributing transistor TG4, a gate electrode 751 of distributing transistor TG5, a gate electrode 761 of distributing transistor TG6, a gate electrode 771 of distributing transistor TG7, and a gate electrode 781 of charge discharging transistor OFG are arranged radially from the center. The gate electrodes 701 to 771 of distributing transistors TG and the gate electrode 781 of charge discharging transistor OFG are arranged so as to overlap at least a portion of the photodiode PD made of an N-type semiconductor region.

[0142] The distribution transistor TG0 is provided with a charge storage unit MEM0 provided with an electrode 702 and a transfer transistor TRG0 provided with a gate electrode 703. The distribution transistors TG1 to TG7 are similar to the distribution transistor TG0.

[0143] 27, the charge drain transistor OFG provided with a gate electrode 781, the reset transistor RST provided with a gate electrode 783, the amplifier transistor AMP provided with a gate electrode 784, and the select transistor SEL provided with a gate electrode 785 are the same as the charge drain transistor OFG, the reset transistor RST, the amplifier transistor AMP, and the select transistor SEL in Fig. 21. That is, the charges accumulated in the floating diffusion regions FD0 to FD7 are converted into a voltage by the amplifier transistor AMP under the control of the select transistor SEL, and output to the vertical signal line (VSL). The reset transistor RST resets the charges accumulated in the floating diffusion regions FD0 to FD7.

[0144] <Light Source Waveform and Driving Voltage Waveform> Fig. 28 is a diagram showing examples of a light source waveform and a driving voltage waveform used in the tenth embodiment. In Fig. 28, LS indicates the light source waveform of the irradiated light IL emitted from the light source 21. The irradiated light IL has a waveform shape in which a first modulation frequency (Fmod1), a second modulation frequency (Fmod2), and a third modulation frequency (Fmod3) are superimposed. The first modulation frequency (Fmod1) is a higher frequency than the second modulation frequency (Fmod2). The second modulation frequency (Fmod2) is a higher frequency than the third modulation frequency (Fmod3).

[0145] In Fig. 28, TG0 indicates the drive voltage waveform of the distribution transistor TG0 of the pixel 31 of Fig. 27, TG1 indicates the drive voltage waveform of the distribution transistor TG1 of the pixel 31 of Fig. 27, TG2 indicates the drive voltage waveform of the distribution transistor TG2 of the pixel 31 of Fig. 27, and TG3 indicates the drive voltage waveform of the distribution transistor TG3 of the pixel 31 of Fig. 27. Also in Fig. 28, TG4 indicates the drive voltage waveform of the distribution transistor TG4 of the pixel 31 of Fig. 27, TG5 indicates the drive voltage waveform of the distribution transistor TG5 of the pixel 31 of Fig. 27, TG6 indicates the drive voltage waveform of the distribution transistor TG6 of the pixel 31 of Fig. 27, and TG7 indicates the drive voltage waveform of the distribution transistor TG7 of the pixel 31 of Fig. 27.

[0146] 27 , within one period of the third modulation frequency (Fmod3), the distribution transistors TG0 and TG1 and the distribution transistors TG2 and TG3 are alternately driven in accordance with the period of the second modulation frequency (Fmod2), and after the distribution transistors TG0 and TG1 alternately distribute charge in accordance with the period of the first modulation frequency (Fmod1), the distribution transistors TG2 and TG3 alternately distribute charge in accordance with the period of the first modulation frequency (Fmod1). Also, within one period of the third modulation frequency (Fmod3), the distribution transistors TG4 and TG5 and the distribution transistors TG6 and TG7 are alternately driven in accordance with the period of the second modulation frequency (Fmod2), and after the distribution transistors TG4 and TG5 alternately distribute charge in accordance with the period of the first modulation frequency (Fmod1), the distribution transistors TG6 and TG7 alternately distribute charge in accordance with the period of the first modulation frequency (Fmod1). That is, by applying a voltage to the gate electrodes of the distribution transistors TG0 to TG7 at the timing of the drive voltage waveform in FIG. 28, the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD0 to FD7.

[0147] As described above, in the distance measuring device 1, when the illumination light IL emitted by the light emitting unit 11 has a waveform generated by multiplying N (N is a natural number greater than or equal to 2) different modulation frequencies, a predetermined number of taps can be provided for each pixel arranged in the pixel array unit 61 of the light receiving unit 12, and a drive voltage waveform generated by multiplying the N different modulation frequencies can be applied. In this pixel structure, each tap is formed by a MOS transistor. By adopting such a configuration, the distance measurement range can be expanded.

[0148] 2 is used as the configuration of the distance measuring device 1 in the configuration of the pixel 31 shown in Fig. 10 to Fig. 12 , but a configuration in which two types of polarizers and two corresponding light sources are provided on the irradiation surface side of the pixel 31 may also be used. Next, a configuration in which two light sources are provided in the light source 21 and light from each light source becomes polarization information that matches the two types of polarizers provided on the irradiation surface side of the pixel 31 will be described.

[0149] <Light Source Configuration> Fig. 29 is a diagram showing an example of the configuration of a polarizer installed on the irradiation surface side of a pixel. As shown in Fig. 29, in the light receiving unit 12, polarizers 811-1 and 811-2 are installed on the irradiation surface side of pixels 31-1 and 31-2, which are two pixels arranged adjacent to each other in the row direction (left-right direction in the figure) in the pixel array unit 61. The polarizers 811-1 and 811-2 transmit linearly polarized light with different polarization directions. Here, when the angle between the reference direction and the polarization direction is defined as the polarization angle, the polarization angle of polarizer 811-1 is 45°, and the polarization angle of polarizer 811-2 is 135°.

[0150] <Pixel Structure> Fig. 30 is a diagram showing an example of the planar structure of a pixel in the eleventh embodiment. Fig. 30 shows pixels 31-1 and 31-2, which are two pixels arranged adjacent to each other in the row direction (left-right direction in the figure) in the pixel array section 61. Pixels 31-1 and 31-2 are configured in the same manner as pixel 31 shown in Fig. 10. In Fig. 30, a polarizer 811-1 (Fig. 29) is provided on the irradiation surface side of pixel 31-1, and a polarizer 811-2 (Fig. 29) is provided on the irradiation surface side of pixel 31-2.

[0151] <Light Source Waveform and Driving Voltage Waveform> Fig. 31 is a diagram showing examples of a light source waveform and a driving voltage waveform used in the eleventh embodiment. In Fig. 31, LS0 (45°) indicates the light source waveform of the illumination light IL emitted from one light source LS0 of the two light sources LS0 and LS1 provided in the light source 21. LS1 (135°) indicates the light source waveform of the illumination light IL emitted from the other light source LS1 of the two light sources LS0 and LS1 provided in the light source 21. The illumination light IL emitted from each of the two light sources has a waveform formed by superimposing a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod), but the illumination light IL has waveforms that are 180° inverted from each other.

[0152] In Figure 31, TGO denotes the drive voltage waveform of the distribution transistor TG0 of the pixels 31-1 and 31-2 of Figure 30, and TG1 denotes the drive voltage waveform of the distribution transistor TG1 of the pixels 31-1 and 31-2 of Figure 30. The distribution transistors TG0 and TG1 of the pixels 31-1 and 31-2 of Figure 30 operate at a drive voltage corresponding to the low modulation frequency (L_Fmod). Also in Figure 31, TG2 denotes the drive voltage waveform of the distribution transistor TG2 of the pixels 31-1 and 31-2 of Figure 30, and TG3 denotes the drive voltage waveform of the distribution transistor TG3 of the pixels 31-1 and 31-2 of Figure 30. The distribution transistors TG2 and TG3 of the pixels 31-1 and 31-2 of Figure 30 operate at a drive voltage corresponding to the high modulation frequency (H_Fmod).

[0153] 31 , the polarizer 811-1 installed in pixel 31-1 has polarization information that matches that of light source LS0, and therefore, reflected light RL, which is illumination light IL emitted from light source LS0 and reflected by object 2, is incident on pixel 31-1. In pixel 31-1, by applying voltages to the gate electrodes of the distribution transistors TG0, TG1, TG2, and TG3 at the timing of the drive voltage waveforms shown in FIG. 31 , the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD0, FD1, FD2, and FD3. In addition, the polarizer 811-2 installed in pixel 31-2 has polarization information that matches that of light source LS1, and therefore, reflected light RL, which is illumination light IL emitted from light source LS1 and reflected by object 2, is incident on pixel 31-2. In pixel 31-2, the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD0, FD1, FD2, and FD3 by applying a voltage to the gate electrodes of the distribution transistors TG0, TG1, TG2, and TG3 at the timing of the drive voltage waveforms shown in Figure 31. Here, the distance measurement results for low and high frequencies can be calculated from the signals of pixels 31-1 and 31-2 and integrated into one.

[0154] As described above, in the distance measuring device 1, the illumination light IL emitted by the light source 21 (two light sources) of the light-emitting unit 11 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), each containing light corresponding to different polarization information. Two pixels arranged in the pixel array unit 61 of the light-receiving unit 12 can be paired, and two types of polarizers corresponding to the polarization information can be installed on the illumination side of the two pixels. By adopting such a configuration, the amount of light incident on the pixel increases, thereby improving distance measurement accuracy. Note that in this example, the polarization angles are 45° and 135°, but other angles may also be used.

[0155] 1 is an example of the configuration of a distance measuring device, and other configurations may be adopted. For example, the distance measuring device may be provided with a host device, and data and signals may be exchanged between the distance measuring module and the host device.

[0156] <Device Configuration> Fig. 32 is a block diagram showing an example configuration of an embodiment of a distance measuring device to which the present disclosure is applied. In Fig. 32, the distance measuring device 1000 is composed of a distance measuring module 1001 and a host device 1002. The distance measuring module 1001 and the host device 1002 are connected via a predetermined interface. The distance measuring module 1001 performs distance measurement using an indirect ToF method. The distance measuring module 1001 is composed of a light source device 1011 that irradiates an object 2 with irradiation light IL having a predetermined period, and an imaging device 1012 that measures the distance to the object 2 by detecting the phase difference between the irradiation light IL and reflected light RL from the object 2.

[0157] The light source device 1011 is composed of an emission control unit 1021, an emission unit 1022, and a lens 1023. The imaging device 1012 is composed of a lens 1024 and a photodetector 1025. The photodetector 1025 is a light-receiving unit composed of a light-receiving element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The photodetector 1025 generates a control signal and a superimposition signal and supplies them to the emission control unit 1021. The emission control unit 1021 controls the emission unit 1022 based on the control signal and superimposition signal supplied from the photodetector 1025. The emission unit 1022 has a light source that emits infrared light (IR). The emission unit 1022 irradiates the target 2 with irradiation light IL consisting of a light source waveform formed in accordance with control from the emission control unit 1021 via the lens 1023.

[0158] The photodetector element 1025 operates in accordance with a control signal input from the host device 1002. The photodetector element 1025 receives reflected light RL, which is generated by reflecting illumination light IL off the object 2, via a lens 1024. The photodetector element 1025 includes a pixel array in which pixels, each having a photoelectric conversion unit, are arranged two-dimensionally, and generates electric charges based on the intensity of the received light. The photodetector element 1025 applies a voltage having a drive voltage waveform corresponding to the light source waveform to the pixels, acquires a signal, and performs signal processing on the acquired signal to obtain a distance measurement result. The photodetector element 1025 outputs distance measurement data corresponding to the distance measurement result to the host device 1002.

[0159] Here, the configuration of the photodetector element 1025 of FIG. 32 will be described with reference to FIGS. 33 to 37 . FIG. 33 is a diagram showing an example of the configuration of the photodetector element 1025. The photodetector element 1025 can be configured as a chip in which a substrate 1051 on which a photoelectric conversion unit is formed and a substrate 1052 on which a logic circuit is formed are stacked. The inter-substrate connection between the substrates 1051 and 1052 can be achieved using a through-silicon via (TSV), Cu-Cu bonding, or the like. FIG. 34 is a diagram showing an example of the planar configuration of the substrate 1051 of FIG. 33 . In FIG. 34 , a photoelectric conversion unit 1031A such as a photodiode (PD) is two-dimensionally formed on the substrate 1051.

[0160] FIG. 35 is a diagram showing an example of the planar configuration of the substrate 1052 of FIG. 33. In FIG. 35, pixel circuits 1031B that perform analog signal processing are formed two-dimensionally on the substrate 1052. The pixel circuits 1031B are composed of, for example, pixel transistors. That is, on the substrate 1051 of FIG. 34 and the substrate 1052 of FIG. 35, photoelectric conversion units 1031A and pixel circuits 1031B are formed to correspond to each other, and the photoelectric conversion units 1031A and pixel circuits 1031B form pixels 1031. The pixels 1031 configured in this manner are arranged two-dimensionally and form a pixel array unit 1061.

[0161] 35 , a substrate 1052 is provided with a pixel array section 1061, a vertical control section 1071, a pixel light source driving section 1072, a column processing section 1073, a horizontal driving section 1074, and a system control section 1075. In the pixel array section 1061, a drive line is formed for each row of pixel circuits 1031B arranged two-dimensionally (in a matrix). The vertical control section 1071 is composed of a shift register, an address decoder, etc., and controls the pixel circuits 1031B arranged in the pixel array section 1061 by supplying a drive signal via the drive line.

[0162] The pixel light source drive unit 1072 generates a low-frequency signal and a high-frequency signal. The pixel light source drive unit 1072 generates a superimposed signal by superimposing the low-frequency signal and the high-frequency signal, and supplies the superimposed signal to the light emission control unit 1021. The pixel light source drive unit 1072 also supplies the low-frequency signal and the high-frequency signal to each pixel circuit 1031B in the pixel row selected by the vertical control unit 1071 via the drive lines.

[0163] The pixel signals output from each pixel circuit 1031B of the pixel row selected and scanned by the vertical control unit 1071 are supplied to the column processing unit 1073 via the vertical signal lines (VSL). The column processing unit 1073 performs various signal processing on the pixel signals output from each pixel circuit 1031B of the selected pixel row via the vertical signal lines (VSL) for each pixel column of the pixel array unit 1061. For example, the column processing unit 1073 performs AD conversion (Analog to Digital Conversion) and CDS (Correlated Double Sampling) processing. The column processing unit 1073 also performs signal processing such as signal separation and distance measurement result calculation. The distance measurement data obtained by the signal processing by the column processing unit 1073 is output to the host device 1002.

[0164] The horizontal driver 1074 is composed of a shift register, an address decoder, etc., and sequentially selects signal processing circuits corresponding to pixel columns in the column processor 1073. The system controller 1075 is composed of a timing generator that generates various timing signals, etc., and controls the driving of the vertical controller 1071, pixel light source driver 1072, column processor 1073, horizontal driver 1074, etc., based on control signals such as various timing signals. The system controller 1075 also supplies control signals such as various timing signals to the light emission controller 1021.

[0165] Fig. 36 is a diagram showing an example of the configuration of the pixel light source driver 1072 of Fig. 35. In Fig. 36, the pixel light source driver 1072 is made up of a low-frequency signal generator 1081, a high-frequency signal generator 1082, and a modulation signal superimposing unit 1083.

[0166] The low-frequency signal generation unit 1081 generates a low-frequency signal. The high-frequency signal generation unit 1082 generates a high-frequency signal. The high-frequency signal is a signal with a higher frequency than the low-frequency signal. The low modulation frequency (L_Fmod), which is a frequency corresponding to the low-frequency signal, and the high modulation frequency (H_Fmod), which is a frequency corresponding to the high-frequency signal, can be, for example, L_Fmod = 20 MHz and H_Fmod = 200 MHz. In other words, the high modulation frequency (H_Fmod) can be an integer multiple of the low modulation frequency (L_Fmod). The low-frequency signal and the high-frequency signal are supplied to the modulation signal superimposition unit 1083. The modulation signal superimposition unit 1083 superimposes the low-frequency signal supplied from the low-frequency signal generation unit 1081 and the high-frequency signal supplied from the high-frequency signal generation unit 1082, and supplies the superimposed signal to the light-emission control unit 1021 ( FIG. 32 ). The low-frequency signal and the high-frequency signal are supplied to the pixels 1031 (pixel circuits 1031B) arranged in the pixel array section 1061 via drive lines.

[0167] Fig. 37 is a diagram showing an example of the configuration of the column processing unit 1073 in Fig. 35. In Fig. 37, the column processing unit 1073 is composed of a readout unit 1091, a signal separation unit 1092, a low-frequency signal processing unit 1093, a high-frequency signal processing unit 1094, an integration processing unit 1095, and a ranging data transmission unit 1096.

[0168] The readout unit 1091 performs AD conversion and CDS processing on pixel signals output from each pixel 1031 (pixel circuit 1031B) in a selected pixel row via a vertical signal line (VSL) for each pixel column in the pixel array unit 1061, and supplies the signals obtained as a result of the processing to a signal separation unit 1092. The signal separation unit 1092 separates the signals supplied from the readout unit 1091 into signals corresponding to low frequencies and high frequencies. The signal separation unit 1092 supplies the signals corresponding to low frequencies to a low-frequency signal processing unit 1093 and the signals corresponding to high frequencies to a high-frequency signal processing unit 1094.

[0169] The low-frequency signal processing unit 1093 calculates a ranging result for the low frequency based on the signal corresponding to the low frequency supplied from the signal separation unit 1092, and supplies the result to the integration processing unit 1095. The high-frequency signal processing unit 1094 calculates a ranging result for the high frequency based on the signal corresponding to the high frequency supplied from the signal separation unit 1092, and supplies the result to the integration processing unit 1095. The integration processing unit 1095 integrates the ranging result for the low frequency supplied from the low-frequency signal processing unit 1093 and the ranging result for the high frequency supplied from the high-frequency signal processing unit 1094, and supplies the resulting ranging result to the ranging data transmission unit 1096. The ranging data transmission unit 1096 outputs the ranging result supplied from the integration processing unit 1095 to the host device 1002 as ranging data. The ranging data includes ranging results such as distance information.

[0170] <Processing Flow> The processing flow performed by the distance measuring device 1000 in FIG. 32 will be described with reference to the flowchart in FIG.

[0171] In step S111, the low-frequency signal generating unit 1081 generates a low-frequency signal, and the high-frequency signal generating unit 1082 generates a high-frequency signal. In step S112, the modulation signal superimposing unit 1083 superimposes the low-frequency signal and the high-frequency signal. In step S113, the light emitting unit 1022, under the control of the light emission control unit 1021, emits illumination light IL having a modulated waveform from the light source toward the target 2.

[0172] In step S114, the pixel 1031 (the photoelectric conversion unit 1031A) arranged in the pixel array unit 1061 of the photodetector element 1025 receives reflected light RL from the object 2. In step S115, the readout unit 1091 performs AD conversion and CDS processing on the pixel signal output from the pixel 1031 (the pixel circuit 1031B). In step S116, the signal separation unit 1092 separates the signal acquired by the readout unit 1091 into signals corresponding to low frequencies and high frequencies.

[0173] In step S117, the low-frequency signal processing unit 1093 calculates the distance measurement result for the low frequency, and the high-frequency signal processing unit 1094 calculates the distance measurement result for the high frequency. In step S118, the integration processing unit 1095 integrates the two distance measurement results for the low frequency and the high frequency into one. That is, each signal processing unit performs signal processing individually for the low frequency and the high frequency, converts the separated signals into distance information, and integrates the distance information obtained by each signal processing to obtain true distance information. In step S119, the distance measurement data transmission unit 1096 outputs the distance measurement data obtained by integrating the two distance measurement results to the host device 1002.

[0174] <Superimposed signal> Fig. 39 is a diagram showing an example of a superimposed signal of a low frequency signal and a high frequency signal. As shown in Fig. 39, the superimposed signal generated by the modulated signal superimposing unit 1083 is generated by multiplying a low frequency signal by a high frequency signal. The low frequency signal is a signal with a frequency corresponding to the low modulation frequency (L_Fmod), and its period is T L The high frequency signal is a signal with a frequency corresponding to the high modulation frequency (H_Fmod), and its period is T H The period is T L >Period T H As shown in FIG. 39, in the superimposed signal in which a low frequency signal and a high frequency signal are superimposed, the period T L The H level part of the square wave corresponding to the period T H It is a square wave corresponding to

[0175] <Pixel Structure> Fig. 40 is a diagram showing an example of the planar structure of a pixel in the twelfth embodiment. As shown in Fig. 40, the planar structure of a pixel 1031 is the same as the planar structure of the pixel 31 shown in Fig. 10. However, for convenience of explanation, in Fig. 40, the distribution transistors TG0 and TG1 on the inside of Fig. 10 are referred to as distribution transistors TGA and TGB, and the distribution transistors TG2 and TG3 on the outside of Fig. 10 are referred to as distribution transistors TGC and TGD. In addition, with the change in the names of the distribution transistors, the charge storage units MEM0, MEM1, MEM2, MEM3 are now called charge storage units MEM_AC, MEM_AD, MEM_BC, MEM_BD, the transfer transistors TRG0, TRG1, TRG2, TRG3 are now called transfer transistors TRG_AC, TRG_AD, TRG_BC, TRG_BD, and the floating diffusion regions FD0, FD1, FD2, FD3 are now called floating diffusion regions FD_AC, FD_AD, FD_BC, FD_BD.

[0176] Fig. 41 is a diagram showing an example of the waveforms of light and drive voltage during distance measurement. In Fig. 41, IL indicates the waveform of irradiated light IL during distance measurement, and RL indicates the waveform of reflected light RL during distance measurement. The irradiated light IL and reflected light RL have waveforms formed by superimposing a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod), but the reflected light RL is delayed in time because it is irradiated light IL reflected by the object 2.

[0177] In Fig. 41, TGA denotes the drive voltage waveform of the distribution transistor TGA of the pixel 1031 in Fig. 40, and TGB denotes the drive voltage waveform of the distribution transistor TGB of the pixel 1031 in Fig. 40. The distribution transistors TGA and TGB of the pixel 1031 in Fig. 40 operate with a drive voltage corresponding to a low modulation frequency (L_Fmod). The level of the drive voltage waveform of the distribution transistor TGB is inverted relative to the level of the drive voltage waveform of the distribution transistor TGA (they are 180° out of phase with each other), and the charge accumulated in the photodiode PD is distributed to the charge accumulation units MEM_AC and MEM_AD and the charge accumulation units MEM_BC and MEM_BD in accordance with the low modulation frequency (L_Fmod).

[0178] 41, TGC denotes the drive voltage waveform of the distribution transistor TGC of the pixel 1031 in FIG. 40, and TGD denotes the drive voltage waveform of the distribution transistor TGD of the pixel 1031 in FIG. 40. The distribution transistors TGC and TGD of the pixel 1031 in FIG. 40 operate with a drive voltage corresponding to the high modulation frequency (H_Fmod). The level of the drive voltage waveform of the distribution transistor TGD is inverted relative to the level of the drive voltage waveform of the distribution transistor TGC (they are 180° out of phase with each other), and the charge accumulated in the photodiode PD is distributed to the charge accumulation units MEM_AC and MEM_BC and the charge accumulation units MEM_AD and MEM_BD in accordance with the high modulation frequency (H_Fmod).

[0179] Fig. 42 is a diagram showing an example of wiring in the pixel array unit 1061 of Fig. 35. In Fig. 42, drive lines 1032A and 1032B are formed for each pixel row with respect to pixel circuits 1031B arranged two-dimensionally in the pixel array unit 1061. The drive lines 1032A and 1032B for each pixel row are electrically connected to the pixel light source drive unit 1072 (Fig. 35), and supply low-frequency signals (LS) and high-frequency signals (HS) from the pixel light source drive unit 1072 to the distribution transistors TG of each pixel circuit 1031B via contacts 1033A and 1033B formed in each pixel circuit 1031B.

[0180] Specifically, a low-frequency signal (LS) from the pixel light source drive unit 1072 is supplied to the gate electrode 201 of the distribution transistor TGA and the gate electrode 211 of the distribution transistor TGB in each pixel circuit 1031B via a contact 1033A electrically connected to a drive line 1032A. Also, a high-frequency signal (HS) from the pixel light source drive unit 1072 is supplied to the gate electrodes 202 and 212 of the distribution transistor TGC and the gate electrodes 206 and 216 of the distribution transistor TGD in each pixel circuit 1031B via a contact 1033B electrically connected to a drive line 1032B.

[0181] 40, three pairs of taps are provided, with the gate electrode 201 of the distributing transistor TGA and the gate electrode 211 of the distributing transistor TGB being a first pair, the gate electrode 202 of the distributing transistor TGC and the gate electrode 206 of the distributing transistor TGD being a second pair, and the gate electrode 212 of the distributing transistor TGC and the gate electrode 216 of the distributing transistor TGD being a third pair. In the pixel 1031 of FIG. 40, charge is distributed in two stages by the first pair of taps on the inside and the second or third pair of taps on the outside.

[0182] <Phase-Shifted Distance Measurement> When performing distance measurement using the indirect ToF method, the distance measuring device 1000 can increase distance measurement information by shifting the phase to improve accuracy and extend the distance. FIG. 43 is a diagram showing an example of a drive voltage waveform when performing distance measurement with a phase shift. For example, as shown in FIG. 43, distance measurement is performed with a phase shift of 90°. That is, light reception with a phase delay of 0°, which is received without any phase shift from the light receiving side, is used as a reference (reference time t1), and light reception with a phase delay of 90°, a phase delay of 180°, and a phase delay of 270° is performed.

[0183] In Figure 43, TGA0, TGB0, TGC0, and TGD0 indicate drive voltage waveforms of the distribution transistors TGA, TGB, TGC, and TGD in Figure 40 during distance measurement with a phase delay of 0° (hereinafter also referred to as Phase 0°), and TGA90, TGB90, TGC90, and TGD90 indicate drive voltage waveforms of the distribution transistors TGA, TGB, TGC, and TGD in Figure 40 during distance measurement with a phase delay of 90° (hereinafter also referred to as Phase 90°). Also in Figure 43, TGA180, TGB180, TGC180, and TGD180 indicate drive voltage waveforms of the distribution transistors TGA, TGB, TGC, and TGD in Figure 40 during distance measurement with a phase delay of 180° (hereinafter also referred to as Phase 180°), and TGA270, TGB270, TGC270, and TGD270 indicate drive voltage waveforms of the distribution transistors TGA, TGB, TGC, and TGD in Figure 40 during distance measurement with a phase delay of 270° (hereinafter also referred to as Phase 270°).

[0184] As shown in Fig. 43, even when distance measurement is performed with a phase shift of 90°, the distribution transistors TGA and TGB on the inside of Fig. 40 operate with a drive voltage corresponding to the low modulation frequency (L_Fmod), and the distribution transistors TGC and TGD on the outside of Fig. 40 operate with a drive voltage corresponding to the high modulation frequency (H_Fmod). In other words, the distribution transistor TG operates with a phase shift of 90° for each cycle of the low modulation frequency (L_Fmod) and the high modulation frequency (H_Fmod).

[0185] Figure 44 shows an example of a sequence for phase-shifted ranging. When performing ranging using two modulation frequencies (Fmod), low frequency and high frequency, the measurement start times for the low frequency and high frequency are aligned, and low frequency and high frequency signals are acquired simultaneously. Therefore, Figure 44 shows the ranging sequences for low frequency and high frequency on the same time axis. As shown in Figure 44, in low frequency ranging, the ranging period for one frame consists of periods Ql0, Ql1, Ql2, and Ql3. Each period consists of an integration period and a readout period.

[0186] During the integration period of the period Q10, the signal S 0_Low During the integration period Q11, the signal S 90_Low During the integration period of the period Q12, the signal S 180_Low During the integration period of the period Q13, the signal S 270_Low is obtained.

[0187] In high-frequency ranging, the ranging period for one frame is periods Qh0, Qh1, Qh2, and Qh3, which correspond to periods Ql0, Ql1, Ql2, and Ql3, and low-frequency and high-frequency signals are acquired simultaneously.

[0188] During the integration period Qh0, the signal S 0_High During the integration period of the period Qh1, the signal S 90_High During the integration period of the period Qh2, the signal S 180_High During the integration period of the period Qh3, the signal S 270_High is obtained.

[0189] <Circuit Configuration and Operation of Pixel> Figure 45 is a diagram showing an example of the circuit configuration of the pixel 1031 in Figure 40. As shown in Figure 45, the pixel 1031 has a photodiode PD as a photoelectric conversion unit 1031A and a charge discharging transistor OFG. One of the source and drain of the charge discharging transistor OFG is electrically connected to the photodiode PD. The other of the source and drain of the charge discharging transistor OFG is electrically connected to a power supply potential VDD. The charge discharging transistor OFG becomes conductive in response to a voltage applied to its gate, and is able to discharge the charge accumulated in the photodiode PD.

[0190] The pixel 1031 also has a distribution transistor TGA, distribution transistors TGC and TGD, charge storage units MEM_AC and MEM_AD, transfer transistors TRG_AC and TRG_AD, a distribution transistor TGB, distribution transistors TGC and TGD, charge storage units MEM_BC and MEM_BD, and transfer transistors TRG_BC and TRG_BD.

[0191] One of the source or drain of the distributing transistor TGA is electrically connected to the photodiode PD. The other of the source or drain of the distributing transistor TGA is electrically connected to one of the source or drain of the distributing transistors TGC and TGD. The other of the source or drain of the distributing transistor TGC is electrically connected to the charge storage unit MEM_AC. The other of the source or drain of the distributing transistor TGD is electrically connected to the charge storage unit MEM_AD. The distributing transistor TGA and the distributing transistors TGB and TGC are turned on in response to the voltage applied to their gates, and can transfer the charge stored in the photodiode PD to the charge storage units MEM_AC and MEM_AD, respectively.

[0192] One of the source or drain of the transfer transistor TRG_AC is electrically connected to the charge storage unit MEM_AC, and the other of the source or drain is electrically connected to the floating diffusion region FD_AC. The transfer transistor TRG_AC becomes conductive in response to a voltage applied to its gate, and can transfer the charge stored in the charge storage unit MEM_AC to the floating diffusion region FD_AC. One of the source or drain of the transfer transistor TRG_AD is electrically connected to the charge storage unit MEM_AD, and the other of the source or drain is electrically connected to the floating diffusion region FD_AD. The transfer transistor TRG_AD becomes conductive in response to a voltage applied to its gate, and can transfer the charge stored in the charge storage unit MEM_AD to the floating diffusion region FD_AD. The same applies to the distribution transistor TGB, distribution transistors TGC and TGD, charge storage units MEM_BC and MEM_BD, and transfer transistors TRG_BC and TRG_BD.

[0193] The pixel 1031 further includes a reset transistor RST, an amplifier transistor AMP, and a select transistor SEL. The floating diffusion regions FD_AC and FD_AD and the floating diffusion regions FD_BC and FD_BD are electrically connected to the gate of the amplifier transistor AMP, which converts electric charges into voltages and outputs the converted signals. One of the source and drain of the amplifier transistor AMP is electrically connected to one of the source and drain of the select transistor SEL, which outputs the converted signals to a vertical signal line (VSL) in accordance with a selection signal. The other of the source and drain of the amplifier transistor AMP is electrically connected to a power supply potential VDD. The other of the source and drain of the select transistor SEL is electrically connected to the vertical signal line (VSL) and a constant current source 226, and is further electrically connected to the readout unit 1091 ( FIG. 37 ) of the column processing unit 1073. The gate of the select transistor SEL is connected to a drive line that selects a pixel row from which a signal is to be output, and is electrically connected to the vertical control unit 1071 ( FIG. 35 ).

[0194] That is, the charges accumulated in the floating diffusion regions FD_AC, FD_AD and the floating diffusion regions FD_BC, FD_BD are converted into voltages by the amplifier transistor AMP under the control of the select transistor SEL and output to the vertical signal line (VSL). The floating diffusion regions FD_AC, FD_AD and the floating diffusion regions FD_BC, FD_BD are electrically connected to one of the source and drain of a reset transistor RST, which resets the accumulated charges. The gate of the reset transistor RST is electrically connected to a drive line and to the vertical control unit 1071 (FIG. 35). The other of the source and drain of the reset transistor RST is electrically connected to the power supply potential VDD. The reset transistor RST becomes conductive in response to the voltage applied to its gate, and can reset the charges accumulated in the floating diffusion regions FD_AC, FD_AD and the floating diffusion regions FD_BC, FD_BD. Note that the circuit diagram shown in Figure 45 is an example of an equivalent circuit of pixel 1031 in Figure 40, and is not limited to the example shown in Figure 45, and may include, for example, other elements, and is not particularly limited.

[0195] Figure 46 is a timing chart including a signal readout operation in the equivalent circuit of the pixel 1031 shown in Figure 45. Figure 46 shows the operation during the integration period Ta and the readout period Tb. During the integration period Ta, reflected light RL having a waveform in which a low modulation frequency and a high modulation frequency are superimposed is received by the photodiode PD, and the distribution transistors TGA and TGB operate at a drive voltage corresponding to the low modulation frequency, while the distribution transistors TGC and TGD operate at a drive voltage corresponding to the high modulation frequency. As a result, charges are accumulated in the charge storage unit MEMX, i.e., in each of the charge storage units MEM_AC, MEM_AD, MEM_BC, and MEM_BD.

[0196] During the readout period Tb, the transfer transistors TRG_AC, TRG_AD, TRG_BC, and TRG_BD are turned on (conductive in response to an H-level voltage) in that order, causing the charges stored in the charge storage units MEM_AC, MEM_AD, MEM_BC, and MEM_BD to be transferred sequentially. In the equivalent circuit of the pixel 1031 shown in Figure 45, one source follower amplifier is shared by the four FDs, namely, the floating diffusion regions FD_AC, FD_AD, FD_BC, and FD_BD, and therefore readout is performed sequentially.

[0197] Specifically, when the reset transistor RST is turned on, the floating diffusion region FD_AC is initialized, and immediately after the initialization is completed, the selection transistor SEL is turned on, thereby AD-converting the reset level (hereinafter also referred to as the P-phase level). Subsequently, the transfer transistor TRG_AC is turned on to transfer the charge in the charge storage unit MEM_AC to the floating diffusion region FD_AC. Immediately after the transfer is completed, the selection transistor SEL is turned on, thereby AD-converting the signal level (hereinafter also referred to as the D-phase level). CNV indicates the read timing in the readout unit 1091 (FIG. 37), and DOUT indicates the data obtained by AD-converting the signal applied to the vertical signal line (VSL) by the readout unit 1091. The digital signal obtained by AD-converting the P-phase level is also referred to as P-phase data. The digital signal obtained by AD-converting the D-phase level is also referred to as D-phase data. The readout unit 1091 performs CDS processing to determine the difference between the P-phase data (Dac_p) and the D-phase data (Dac_d) as net pixel data.

[0198] Similarly, after the reset transistor RST initializes the floating diffusion region FD_AD, the transfer transistor TRG_AD transfers the charge in the charge storage unit MEM_AD to the floating diffusion region FD_AD, thereby performing AD conversion on the P-phase level and the D-phase level. Then, a CDS process is performed to obtain the difference between the P-phase data (Dad_p) and the D-phase data (Dad_d) as net pixel data. Furthermore, after the reset transistor RST initializes the floating diffusion region FD_BC, the transfer transistor TRG_BC transfers the charge in the charge storage unit MEM_BC to the floating diffusion region FD_BC, thereby performing AD conversion on the P-phase level and the D-phase level. Then, a CDS process is performed to obtain the difference between the P-phase data (Dbc_p) and the D-phase data (Dbc_d) as net pixel data. Furthermore, after the reset transistor RST initializes the floating diffusion region FD_BD, the transfer transistor TRG_BD transfers the charge in the charge storage unit MEM_BD to the floating diffusion region FD_BD, thereby performing AD conversion on the P-phase level and the D-phase level. Then, CDS processing is performed to obtain the difference between the P-phase data (Dbd_p) and the D-phase data (Dbd_d) as net pixel data. After that, the reset transistor RST is turned on, and then the charge discharge transistor OFG is turned on, thereby discharging the charge accumulated in the photodiode PD.

[0199] <Processing in Signal Separation Unit> Fig. 47 is a diagram showing a list of signals (data) output from the readout unit 1091 to the signal separation unit 1092 in the column processing unit 1073 in Fig. 37. Fig. 47 shows signals (data names) obtained by performing AD conversion and CDS processing by the readout unit 1091 on signals obtained from the charges of four FDs for each phase when ranging is performed with the phase shifted by 90°.

[0200] As shown in FIG. 47, in Phase 0°, the readout unit 1091 performs AD conversion and CDS processing on the signal obtained from the charge of the floating diffusion region FD_AC, thereby generating a signal Q AC_0is output to the signal separator 1092. Similarly, the signals obtained from the charges of the floating diffusion regions FD_AD, FD_BC, and FD_BD are subjected to AD conversion and CDS processing, thereby generating a signal Q AD_0 , Q BC_0 , Q BD_0 is obtained.

[0201] Similarly, at Phase 90°, the signals obtained from the charges of the floating diffusion regions FD_AC, FD_AD, FD_BC, and FD_BD are subjected to AD conversion and CDS processing, resulting in signal Q AC_90 , Q AD_90 , Q BC_90 , Q BD_90 In addition, in Phase 180°, the signals obtained from the charges of the floating diffusion regions FD_AC, FD_AD, FD_BC, and FD_BD are subjected to AD conversion and CDS processing, resulting in the signal Q AC_180 , Q AD_180 , Q BC_180 , Q BD_180 Furthermore, in Phase 270°, the signals obtained from the charges of the floating diffusion regions FD_AC, FD_AD, FD_BC, and FD_BD are subjected to AD conversion and CDS processing, resulting in signal Q AC_270 , Q AD_270 , Q BC_270 , Q BD_270 is obtained.

[0202] 47 is input from the readout unit 1091 to the signal separation unit 1092, which uses the input signal to separate signals corresponding to low frequencies from signals corresponding to high frequencies. For example, the signal separation unit 1092 can separate signals corresponding to low frequencies using the following equation (6). The signal separation unit 1092 can also separate signals corresponding to high frequencies using the following equation (7).

[0203]

[0204]

[0205] In equation (6), S 0_Low , S90_Low , S 180_Low , S 270_Low represents a signal corresponding to the low frequency for each phase, and these signals are output to the low frequency signal processing unit 1093. Also, in equation (7), S 0_High , S 90_High , S 180_High , S 270_High represents a signal corresponding to the high frequency for each phase, and these signals are output to the high frequency signal processing unit 1094.

[0206] <Processing of the signal processing unit> The low-frequency signal processing unit 1093 in FIG. 37 receives the signal S corresponding to the low frequency for each phase from the signal separation unit 1092. 0_Low , S 90_Low , S 180_Low , S 270_Low Using this, the distance measurement result θ l The low-frequency signal processing unit 1093 calculates the distance measurement result θ using, for example, the following equation (8): l can be calculated.

[0207]

[0208] A of Fig. 48 is a graph showing the relationship between the phase difference and distance in the case of low frequency. In A of Fig. 48, the vertical axis represents the phase difference, and the horizontal axis represents the distance. As shown in A of Fig. 48, in the case of low frequency, assuming that the distance that can be measured in one cycle is 0 to 2.5 (m), the distance D corresponding to the phase difference θ = 0 to 2π is 0 to 2.5 (m). Assuming that the phase difference θ is l When the phase difference θ l The candidate distances obtained from this are distances D11, D12, and D13. In other words, in the case of a low frequency, candidate distances are obtained that are natural number times the distance obtained from the phase, so the candidate distances are p × f(fmod, θ l ) where p is a natural number and f() is a function that calculates the distance from the frequency fmod and the phase difference θ.

[0209] The high frequency signal processing unit 1094 in FIG. 37 receives the signal S corresponding to the high frequency for each phase from the signal separation unit 1092. 0_High, S 90_High , S 180_High , S 270_High Using this, the distance measurement result θ h The high frequency signal processing unit 1094 calculates the distance measurement result θ using, for example, the following equation (9): h can be calculated.

[0210]

[0211] Fig. 48B is a graph showing the relationship between the phase difference and distance in the case of high frequency. In Fig. 48B, the vertical axis represents the phase difference, and the horizontal axis represents the distance. As shown in Fig. 48B, in the case of high frequency, assuming that the distance that can be measured in one cycle is 0 to 1.5 (m), the distance D corresponding to the phase difference θ = 0 to 2π is 0 to 1.5 (m). Assuming that the phase difference θ is h When the phase difference θ h The candidate distances obtained from the above are distances Dh1, Dh2, Dh3, Dh4, and Dh5. In other words, in the case of high frequency, candidate distances are obtained as natural number times the distance obtained from the phase, so the candidate distances are q × f(fmod, θ h ) where q is a natural number.

[0212] <Processing of Integration Processor> The integration processor 1095 in Fig. 37 integrates the distance measurement results using the processing results input from the low-frequency signal processor 1093 and the high-frequency signal processor 1094. For example, Fig. 49 shows the low-frequency graph shown in Fig. 48A and the high-frequency graph shown in Fig. 48B superimposed on each other. In Fig. 49, similar to Fig. 48A, the phase difference θ is used as the distance measurement result for the low frequency. l When calculated, the phase difference θ l As the candidate distances specified by the above, distances D11, D12, and D13 are obtained. As in B of FIG. 48, the phase difference θ h When calculated, the phase difference θ h As candidate distances specified by the above, distances Dh1, Dh2, Dh3, Dh4, and Dh5 are obtained.

[0213] When there are multiple candidates for the distance measurement result, a single result can be obtained from the multiple candidates by comparing the low-frequency candidate distance with the high-frequency candidate distance and determining the candidate distance that satisfies a predetermined condition. For example, when distance D11 is compared with distances Dh1, Dh2, Dh3, Dh4, and Dh5, distance Dh1 is left as the candidate distance because the difference value A between distance D11 and distance Dh1 is the smallest. When distance D12 is compared with distances Dh1, Dh2, Dh3, Dh4, and Dh5, distance Dh3 is left as the candidate distance because the difference value B between distance D12 and distance Dh3 is the smallest. When distance D13 is compared with distances Dh1, Dh2, Dh3, Dh4, and Dh5, distance Dh4 is left as the candidate distance because the difference value C between distance D13 and distance Dh4 is the smallest.

[0214] Then, difference values ​​A, B, and C are compared, and the high-frequency candidate distance with the smallest difference value is determined as the ranging result. In the example of Figure 49, difference value A is the smallest, so distance Dh1 is determined from among distances Dh1, Dh3, and Dh4 that remain as candidate distances. By performing the above-described series of processes for each of the pixels 1031 arranged in the pixel array unit 1061, the ranging data transmission unit 1096 can output ranging data in a format that can display, for example, a distance image.

[0215] Fig. 50 is a diagram showing an example of the output format of distance measurement data. As shown in Fig. 50, distance measurement data is composed of a header, distance measurement information, and a footer. The header contains output start recognition information for recognizing the start of distance measurement data output. The distance measurement information contains dictionary-type data linking coordinates with distance, such as (x0, y0):z00, (x1, y0):z10, .... The footer contains output end recognition information for recognizing the end of distance measurement data output.

[0216] <Application Example> Figures 51 and 52 are diagrams showing an example of volume detection of a moving object by the distance measuring device 1000 of Figure 32. In Figure 51, a distance measuring module 1001 is attached to the tip of an L-shaped member 4, and is capable of measuring the distance from above to an object 2A moving on a belt conveyor 3. When the object 2A on the belt conveyor 3 moves in the direction indicated by arrow M1, and the host device 1002 performs volume calculation using distance measurement data output from the distance measuring module 1001, the calculation results are, for example, as shown in Figure 52. In Figure 52, for comparison, volume calculation using measurement data by a conventional distance measuring device is shown in A of Figure 52, and volume calculation using distance measurement data by the distance measuring device 1000 is shown in B of Figure 52.

[0217] That is, in a conventional distance measuring device, when distance measurement is performed using two modulation frequencies (Fmod), low frequency and high frequency, distance measurements are performed at the low modulation frequency and the high modulation frequency in different frames. In this case, as shown in A of FIG. 52, when the width (true width) of the object 2A is width W, the width erroneously detected due to blur is ΔW, and volume calculation is performed using width W + ΔW, which may result in low accuracy. On the other hand, in the distance measuring device 1000, when distance measurement is performed using two modulation frequencies (Fmod), low frequency and high frequency, distance measurements can be performed at the low modulation frequency and the high modulation frequency in the same frame. In this case, as shown in B of FIG. 52, ΔW relative to the width W of the object 2A can be suppressed, and therefore volume calculation is performed with ΔW suppressed, resulting in more accurate calculation results.

[0218] Fig. 53 is a diagram showing an example of gesture recognition by the distance measuring device 1000 of Fig. 32. For comparison, Fig. 53 shows gesture recognition using measurement data by a conventional distance measuring device in A and B, and gesture recognition using distance measurement data by the distance measuring device 1000 in C. The distance measuring device is installed in the front of the vehicle, such as above the windshield, behind the rearview mirror, or on the dashboard.

[0219] That is, in a conventional distance measuring device, when distance measurement is performed at a low modulation frequency, the distance measurement accuracy decreases as the object moves farther away from the distance measuring device, making it difficult to recognize the gestures of, for example, a person sitting in the back seat of a vehicle. For example, as shown in A of Fig. 53, when a person sitting in the driver's seat and a person sitting in the back seat play rock-paper-scissors, gesture recognition using distance measurement data obtained by distance measurement at a low modulation frequency recognizes that the person in the driver's seat has a paper sign (the hand within frame 1101A), but the hand shape of the person in the back seat cannot be recognized and remains unknown (the hand within frame 1102A).

[0220] Furthermore, in conventional distance measuring devices, when distance measurement is performed at a high modulation frequency, the accuracy of gesture recognition increases, but aliasing may occur, making it impossible to correctly recognize the positional relationship. For example, as shown in B of Fig. 53, gesture recognition using distance measurement data obtained by distance measurement at a high modulation frequency recognizes that the hand shapes of a person inside the car are paper and scissors (the hands within frames 1101B and 1102B), but the positional relationship between the hands is not accurately recognized, making it impossible to determine whether the hand is that of the person in the driver's seat or the person in the back seat.

[0221] On the other hand, by performing distance measurement at low modulation frequencies and high modulation frequencies, the distance measuring device 1000 can compensate for the drawbacks of distance measurement at low modulation frequencies or high modulation frequencies in conventional distance measuring devices, and can recognize who is making what gesture. For example, as shown in C of Fig. 53, gesture recognition using distance measurement data obtained by distance measurement at low modulation frequencies and high modulation frequencies in the same frame makes it possible to recognize what hand signs people sitting where in the car are making, and it is recognized that the person in the driver's seat has a paper sign and the person in the back seat has a scissors sign (hands within frames 1101C and 1102C).

[0222] As described above, the distance measuring device 1000 irradiates the target 2 with illumination light IL having a waveform in which multiple modulation frequencies (Fmod) are superimposed, separates the distance measurement signals obtained from the reflected light RL reflected from the target 2 by modulation frequency (Fmod), and outputs distance measurement data by integrating the distance measurement results obtained from the separated distance measurement signals. For example, the light source provided in the light-emitting unit 1022 is driven based on a superimposed signal in which a low-frequency signal and a high-frequency signal are superimposed, and the pixels 1031 (pixel circuits 1031B) arranged two-dimensionally in the pixel array unit 1061 provided in the photodetector element 1025 as the light-receiving unit are driven based on the low-frequency signal and the high-frequency signal. Here, by using the pixel array wiring shown in FIG. 42, low-frequency and high-frequency signals can be simultaneously applied to one pixel. Furthermore, the column processing unit 1073 separates the distance measurement signals into low-frequency and high-frequency signals, and obtains and integrates the distance measurement results for each frequency. This allows low-frequency and high-frequency signals to be applied simultaneously to one pixel when performing distance measurement using low and high modulation frequencies, which is advantageous for achieving higher resolution. Furthermore, as shown in Figure 44, distance measurement can be performed by aligning the start times of low-frequency and high-frequency signals, allowing low-frequency and high-frequency signals to be acquired simultaneously. Therefore, distance measurement can be performed at low and high frequencies in the same frame, which reduces blurring when measuring the distance to a moving subject.

[0223] 40 has a structure in which three pairs of taps are provided in one pixel 1031, but a structure in which four taps are provided in one pixel 1031 may also be used. Next, a structure in which four taps are provided in one pixel 1031 will be described. The configuration of the distance measuring device 1000 is the same as that in FIG. 32 .

[0224] <Configuration of Logic Circuit Board> Figure 54 is a diagram showing an example of the planar configuration of a substrate 1052 on which a logic circuit is formed. In Figure 54, parts corresponding to those in Figure 35 are given the same reference numerals, and their explanation will be omitted as appropriate. Compared to the substrate 1052 in Figure 35, the substrate 1052 in Figure 54 is provided with a pixel light source driver 1111 instead of the pixel light source driver 1072.

[0225] Fig. 55 is a diagram showing an example of the configuration of the pixel light source driver 1111 of Fig. 54. In Fig. 55, the pixel light source driver 1111 is made up of a low-frequency signal generator 1121, a high-frequency signal generator 1122, and a modulation signal superimposing unit 1123.

[0226] The low-frequency signal generation unit 1121 generates a low-frequency signal and supplies it to the modulation signal superimposition unit 1123. The high-frequency signal generation unit 1122 generates a high-frequency signal and supplies it to the modulation signal superimposition unit 1123. The modulation signal superimposition unit 1123 superimposes the low-frequency signal supplied from the low-frequency signal generation unit 1121 on the high-frequency signal supplied from the high-frequency signal generation unit 1122, and supplies the superimposed signal to the light emission control unit 1021 ( FIG. 32 ) and the pixel circuits 1031B ( FIG. 54 ) arranged in the pixel array unit 1061.

[0227] <Light Source Waveform> FIG. 56 is a diagram showing an example of a light source waveform used in the thirteenth embodiment. As shown in FIG. 56, the illumination light IL emitted from the light-emitting unit 1022 (FIG. 32) has a waveform formed by superimposing a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod). For example, L_Fmod = 20 MHz, H_Fmod = 200 MHz, and a waveform formed by multiplying two different Fmod waveforms can be used as the light source waveform. In the example of the light source waveform shown in FIG. 56, the H-level portion of the rectangular wave corresponding to the L_Fmod period is a rectangular wave corresponding to the H_Fmod period.

[0228] <Pixel Structure> Figure 57 is a diagram showing an example of the planar structure of a pixel in the thirteenth embodiment. As shown in Figure 57, the planar structure of pixel 1031 is the same as the planar structure of pixel 31 shown in Figure 21. However, for convenience of explanation, in Figure 57, the distribution transistors TG0, TG1, TG2, and TG3 of Figure 21 are referred to as distribution transistors TGA, TGB, TGC, and TGD. Furthermore, in accordance with the change in the names of the distribution transistors, the charge storage units MEM0, MEM1, MEM2, and MEM3 are referred to as charge storage units MEM_A, MEM_B, MEM_C, and MEM_D, the transfer transistors TRG0, TRG1, TRG2, and TRG3 are referred to as transfer transistors TRG_A, TRG_B, TRG_C, and TRG_D, and the floating diffusion regions FD0, FD1, FD2, and FD3 are referred to as floating diffusion regions FD_A, FD_B, FD_C, and FD_D.

[0229] <Driving voltage waveform> Fig. 58 is a diagram showing an example of a driving voltage waveform used in the thirteenth embodiment. In Fig. 58, TGA and TGB indicate the driving voltage waveforms of the distribution transistors TGA and TGB of the pixel 1031 of Fig. 57. Also in Fig. 58, TGC and TGD indicate the driving voltage waveforms of the distribution transistors TGC and TGD of the pixel 1031 of Fig. 57.

[0230] 58, the distribution transistors TGA, TGB and the distribution transistors TGC, TGD are alternately driven for each cycle of the low modulation frequency (L_Fmod), and after the distribution transistors TGA, TGB alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod), the distribution transistors TGC, TGD alternately distribute charge in accordance with the cycle of the high modulation frequency (H_Fmod). That is, by applying voltage to the gate electrodes of the distribution transistors TGA, TGB, TGC, TGD at the timing of the drive voltage waveform in Fig. 58, the charge generated in the photodiode PD is transferred to one of the floating diffusion regions FD_A, FD_B, FD_C, FD_D.

[0231] <Example of Wiring> Fig. 59 is a diagram showing an example of wiring in the pixel array unit 1061 of Fig. 54. In Fig. 59, a drive line 1036 is formed for each pixel row for the pixel circuits 1031B arranged two-dimensionally in the pixel array unit 1061. The drive line 1036 for each pixel row is electrically connected to the pixel light source drive unit 1111 (Fig. 54), and supplies the superimposed signal (XS) from the pixel light source drive unit 1111 to the distribution transistor TG of each pixel circuit 1031B via a contact 1037 formed in each pixel circuit 1031B.

[0232] Specifically, the superimposed signal (XS) from the pixel light source drive unit 1111 is supplied to the gate electrode 501 of the distribution transistor TGA, the gate electrode 511 of the distribution transistor TGB, the gate electrode 521 of the distribution transistor TGC, and the gate electrode 531 of the distribution transistor TGD in each pixel circuit 1031B via a contact 1037 electrically connected to the drive line 1036.

[0233] As described above, in the distance measuring device 1000, when the illumination light IL emitted by the light emitting unit 1022 has a waveform generated by multiplying two different modulation frequencies (L_Fmod, H_Fmod), four taps can be provided to each pixel arranged in the pixel array unit 1061 of the photodetector element 1025, and a voltage having a drive voltage waveform formed by multiplying the two different modulation frequencies (L_Fmod, H_Fmod) can be applied. In this case, the distance measurement result using the low modulation frequency (L_Fmod) can be calculated, for example, using the above-mentioned equation (3). Also, the distance measurement result using the high modulation frequency (H_Fmod) can be calculated, for example, using the above-mentioned equation (4). However, in the above-mentioned equations (3) and (4), Q FD0 , Q FD1 , Q FD2 , Q FD3 A, Q FD_A , Q FD_B , Q FD_C , Q FD_D It needs to be rephrased as:

[0234] <<Fourteenth Embodiment>> In the distance measuring device 1000, the distance measuring module 1001 may be operated by switching between a dual mode and a single mode. The single mode is a mode in which distance measurement is performed at one modulation frequency. The dual mode is a mode in which distance measurement is performed at two modulation frequencies.

[0235] <Device Configuration> Fig. 60 is a block diagram showing yet another example configuration of an embodiment of a distance measuring device to which the present disclosure is applied. In Fig. 60, parts corresponding to those in Fig. 32 are assigned the same reference numerals, and their description will be omitted as appropriate. Compared to the host device 1002 in Fig. 32, the host device 1002 in Fig. 60 is provided with an image processing unit 1131 and a memory 1132.

[0236] The image processing unit 1131 is composed of a DSP (Digital Signal Processor) and the like. The image processing unit 1131 performs image processing using distance measurement data input from the photodetector element 1025 of the distance measurement module 1001, and outputs (feeds back) a selection signal according to the processing result to the photodetector element 1025. The selection signal is a signal for selecting a signal to be used in dual mode or single mode. When performing image processing, the image processing unit 1131 temporarily stores data in the memory 1132 as appropriate.

[0237] Figure 61 is a diagram showing an example of the planar configuration of the substrate 1052 in the photodetector element 1025 of Figure 60. In Figure 61, parts corresponding to those in Figure 35 are given the same reference numerals, and their description will be omitted as appropriate. Compared to the substrate 1052 of Figure 35, the substrate 1052 of Figure 61 is provided with a pixel light source drive unit 1141 instead of the pixel light source drive unit 1072. A selection signal is input to the pixel light source drive unit 1141 from the image processing unit 1131 (Figure 60) of the host device 1002.

[0238] Fig. 62 is a diagram showing an example of the configuration of the pixel light source driving unit 1141 of Fig. 61. In Fig. 62, the pixel light source driving unit 1141 is made up of a low-frequency signal generating unit 1151, a high-frequency signal generating unit 1152, a non-periodic signal generating unit 1153, a signal selecting unit 1154, and a modulation signal superimposing unit 1155.

[0239] The low-frequency signal generation unit 1151 generates a low-frequency signal and supplies it to the signal selection unit 1154. The high-frequency signal generation unit 1152 generates a high-frequency signal and supplies it to the modulation signal superimposition unit 1155 and the pixel circuit 1031B (FIG. 61). The non-periodic signal generation unit 1153 generates a non-periodic signal and supplies it to the signal selection unit 1154. A non-periodic signal is a signal that does not have information related to frequency (frequency information).

[0240] The signal selection unit 1154 selects one of the low-frequency signal and the non-periodic signal in accordance with a selection signal input from the image processing unit 1131 (FIG. 60) of the host device 1002, and supplies the selected signal to the modulation signal superimposition unit 1155 and the pixel circuit 1031B (FIG. 61). The modulation signal superimposition unit 1155 superimposes the low-frequency signal or the non-periodic signal supplied from the signal selection unit 1154 on the high-frequency signal supplied from the high-frequency signal generation unit 1152, and supplies the superimposed signal obtained by the superposition to the light emission control unit 1021 (FIG. 60).

[0241] In this way, the pixel light source driving unit 1141 supplies a superimposed signal, which is a superimposed signal of a low-frequency signal or a non-periodic signal and a high-frequency signal, to the light emission control unit 1021 (Figure 60), and supplies the low-frequency signal or a non-periodic signal and the high-frequency signal to the pixels 1031 (pixel circuits 1031B) arranged in the pixel array unit 1061.

[0242] <Comparison of signals for each mode> Fig. 63 is a diagram showing examples of signals that are switched and used for each ranging mode. In Fig. 63, the left side of the figure shows signals used in single mode, and the right side of the figure shows signals used in dual mode.

[0243] When operating in dual mode, the signal selection unit 1154 selects a low-frequency signal, and a superimposed signal obtained by superimposing the low-frequency signal and the high-frequency signal is supplied to the light-emission control unit 1021 ( FIG. 60 ), and the low-frequency signal and the high-frequency signal are supplied to the pixel circuit 1031B ( FIG. 61 ). When operating in single mode, the signal selection unit 1154 selects a non-periodic signal, and a superimposed signal obtained by superimposing the non-periodic signal and the high-frequency signal is supplied to the light-emission control unit 1021 ( FIG. 60 ), and the non-periodic signal and the high-frequency signal are supplied to the pixel circuit 1031B ( FIG. 61 ).

[0244] Here, with regard to the superimposed signal in each mode, when operating in dual mode, a superimposed signal with a superimposed low-frequency signal is used, resulting in periods when no optical pulses are generated (periods when the irradiated light IL is not emitted). Therefore, in dual mode, the inherent high ranging accuracy achieved by using a high modulation frequency (H_Fmod) may not be fully utilized. On the other hand, when operating in single mode, a superimposed signal with a superimposed non-periodic signal is used, resulting in no periods when no optical pulses are generated. Therefore, in the present disclosure, ranging accuracy can be improved by appropriately switching between dual mode and single mode. For example, the image processing unit 1131 supplies a selection signal for selecting a low-frequency signal or a non-periodic signal depending on the results of image processing, thereby obtaining ranging data with improved ranging accuracy depending on the situation.

[0245] <Application Example> Figures 64 and 65 are diagrams showing an example in which the distance measuring device 1000 of Figure 60 is mounted on a surveillance camera. The flow of processing performed by the distance measuring device 1000 mounted on the surveillance camera will be described with reference to the flowchart of Figure 64.

[0246] In step S131, the ranging module 1001 determines whether the ranging mode is the dual mode. If it is determined in step S131 that the ranging mode is not the dual mode, i.e., the single mode, the ranging module 1001 switches the ranging mode to the dual mode (S132). If it is determined in step S131 that the ranging mode is the dual mode, step S132 is skipped and the process proceeds to step S133.

[0247] In step S133, the ranging module 1001 performs ranging in dual mode. In step S134, the image processing unit 1131 performs image processing using ranging data obtained by ranging in dual mode. In step S135, the image processing unit 1131 determines whether a predetermined detection target object (e.g., a human face) has been detected from the distance image. If it is determined in step S135 that the detection target object has been detected, the process proceeds to step S136. In step S136, the image processing unit 1131 stores position information indicating the position of the object on the distance image in the memory 1132.

[0248] In step S137, the image processing unit 1131 outputs a selection signal indicating the selection of single mode to the ranging module 1001 (the light detection element 1025), thereby switching the ranging mode to single mode. In step S138, the ranging module 1001 performs ranging in single mode. In step S139, the image processing unit 1131 uses the position information stored in the memory 1132 to combine the distance image acquired in dual mode and the distance image acquired in single mode. In step S140, the image processing unit 1131 outputs the combined image.

[0249] For example, as shown in A of Fig. 65, when a human face within frame 1161 is detected from a distance image acquired in dual mode, distance measurement is performed in single mode. Then, as shown in B of Fig. 65, by combining the distance image acquired in dual mode and the distance image acquired in single mode, an image including more detailed features of the human face within frame 1161 can be obtained. In this way, an image can be obtained in which distance information is acquired with higher accuracy for a preset object to be detected.

[0250] On the other hand, if it is determined in step S135 that the target object has not been detected, steps S136 to S139 are skipped and the process proceeds to step S140. In step S140, the image processing unit 1131 outputs the distance image acquired in dual mode. When the process of step S140 ends, the series of processes ends.

[0251] Figures 66 to 68 are diagrams showing an example of volume detection of a moving object by the distance measuring device 1000 of Figure 60. The flow of processing performed by the distance measuring device 1000 installed with respect to a moving object will be described with reference to the flowchart of Figure 66.

[0252] In steps S151 to S154, similar to steps S131 to S134 in FIG. 64, the ranging module 1001 performs ranging in dual mode, and the host device 1002 (image processing unit 1131) performs image processing using the ranging data obtained by the ranging in dual mode.

[0253] In step S155, the image processing unit 1131 determines whether a preset detection target object (e.g., luggage) has been detected from the distance image. If it is determined in step S155 that the detection target object has not been detected, the process returns to step S153, and steps S153 to S155 are repeated. If it is determined in step S155 that the detection target object has been detected, the process proceeds to step S156.

[0254] In step S156, the image processing unit 1131 determines whether the thickness of the object detected from the distance image is equal to or less than a predetermined value. If it is determined in step S156 that the thickness of the object is equal to or less than the predetermined value, the process proceeds to step S157. In step S157, the image processing unit 1131 outputs a selection signal indicating the selection of single mode to the distance measurement module 1001 (the light detection element 1025), thereby switching the distance measurement mode to single mode. In step S158, the distance measurement module 1001 performs distance measurement in single mode. In step S159, the image processing unit 1131 performs image processing using the distance measurement data obtained in single mode, and calculates the volume of the object to be detected (e.g., a thin package such as a cardboard box for a CD).

[0255] For example, as shown in FIG. 67 , when an object 2B placed on a belt conveyor 3 moves in the direction indicated by arrow M2, and volume calculation is performed using distance measurement data output from a distance measurement module 1001 attached to the tip of a member 4, the distance measurement mode is switched as shown in FIG. 68 . That is, as shown in FIG. 68A , if an object 2B (e.g., a cardboard box for a CD with a thickness less than a predetermined value) within a frame 1171 is detected from a distance image acquired in dual mode, distance measurement is performed in single mode. Then, as shown in FIG. 68B , by calculating the volume of object 2B from a distance image acquired in single mode, volume detection can be performed using more accurate information, thereby improving estimation accuracy. Note that when a target object is detected, the condition for whether the object satisfies a predetermined condition is not limited to the object's thickness, and one or more other conditions, such as width or shape, may be set. When the predetermined condition is met, the distance measurement mode is switched to single mode.

[0256] On the other hand, if it is determined in step S156 that the thickness of the object exceeds the predetermined value, steps S157 and S158 are skipped and the process proceeds to step S159. In step S159, the image processing unit 1131 performs image processing using the distance measurement data obtained in dual mode to calculate the volume of the object to be detected (e.g., a large piece of luggage). For example, in the case of a large piece of luggage, sufficient estimation accuracy can be obtained even when distance measurement data obtained in dual mode is used. When the process of step S159 is completed, the series of processes ends.

[0257] <Advantages of Mode Switching> The advantages of switching between the dual mode and the single mode will be described with reference to FIGS.

[0258] Figure 69 is a diagram showing an example of the planar structure of a pixel when distance measurement is performed at one modulation frequency. As shown in Figure 69, the planar structure of a pixel when distance measurement is performed at one modulation frequency is the same as the planar structure of pixel 31-1 or pixel 31-2 shown in Figure 7. However, for convenience of explanation, in Figure 69, the distribution transistors TG0 and TG2 of Figure 7 are referred to as distribution transistors TGA, and the distribution transistors TG1 and TG3 of Figure 7 are referred to as distribution transistors TGB. Furthermore, due to changes in the names of the distribution transistors, the charge storage units MEM0 and MEM2 are referred to as charge storage unit MEM_A, the charge storage units MEM1 and MEM3 are referred to as charge storage unit MEM_B, the transfer transistors TRG0 and TRG2 are referred to as transfer transistors TRG_A, the transfer transistors TRG1 and TRG3 are referred to as transfer transistors TRG_B, the floating diffusion regions FD0 and FD2 are referred to as floating diffusion region FD_A, and the floating diffusion regions FD1 and FD3 are referred to as floating diffusion region FD_B.

[0259] Fig. 70 is a diagram showing the waveforms of reflected light RL and drive voltage when distance measurement is performed at one modulation frequency (L_Fmod). In Fig. 70, RL indicates the waveform of reflected light RL when only the low modulation frequency (L_Fmod) is used. TGA and TGB indicate the drive voltage waveforms of the distribution transistors TGA and TGB in Fig. 69. As shown in Fig. 70, the level of the drive voltage waveform of the distribution transistor TGB is inverted with respect to the level of the drive voltage waveform of the distribution transistor TGA, and the charge accumulated in the photodiode PD is distributed to the charge accumulation unit MEM_A side and the charge accumulation unit MEM_B side according to the low modulation frequency (L_Fmod).

[0260] Fig. 71 is a diagram showing the waveforms of reflected light RL and drive voltage when distance measurement is performed at one modulation frequency (H_Fmod). In Fig. 71, RL indicates the waveform of reflected light RL when only the high modulation frequency (H_Fmod) is used. TGA and TGB indicate the drive voltage waveforms of the distribution transistors TGA and TGB in Fig. 69. As shown in Fig. 71, the level of the drive voltage waveform of the distribution transistor TGB is inverted with respect to the level of the drive voltage waveform of the distribution transistor TGA, and the charge accumulated in the photodiode PD is distributed to the charge accumulation unit MEM_A side and the charge accumulation unit MEM_B side in accordance with the high modulation frequency (H_Fmod).

[0261] FIG. 72 is a diagram showing an example of the planar structure of a pixel when distance measurement is performed by superimposing two modulation frequencies (L_Fmod, H_Fmod). As shown in FIG. 72, the planar structure of a pixel when distance measurement is performed by superimposing two modulation frequencies is the same as the planar structure of the pixel 1031 shown in FIG. 40. FIG. 73 is a diagram showing the waveforms of reflected light RL and driving voltage when distance measurement is performed by superimposing two modulation frequencies. In FIG. 73, RL indicates the waveform of reflected light RL when a low modulation frequency (L_Fmod) and a high modulation frequency (H_Fmod) are superimposed. TGA, TGB, TGC, and TGD indicate the driving voltage waveforms of the distribution transistors TGA, TGB, TGC, and TGD in FIG. 72.

[0262] 73, the level of the drive voltage waveform of the distribution transistor TGB is inverted with respect to the level of the drive voltage waveform of the distribution transistor TGA, and the charge accumulated in the photodiode PD is distributed to the charge storage units MEM_AC, MEM_AD and the charge storage units MEM_BC, MEM_BD in accordance with the low modulation frequency (L_Fmod). Also, the level of the drive voltage waveform of the distribution transistor TGD is inverted with respect to the level of the drive voltage waveform of the distribution transistor TGC, and the charge accumulated in the photodiode PD is distributed to the charge storage units MEM_AC, MEM_BC and the charge storage units MEM_AD, MEM_BD in accordance with the high modulation frequency (H_Fmod).

[0263] Here, looking at the period T3 in Fig. 73, when the low modulation frequency (L_Fmod) and the high modulation frequency (H_Fmod) are superimposed, there is a period in which no light pulse is generated (a period in which the irradiated light IL is not emitted), as shown by the dashed rectangle. This is clear from the relationship between the period T1 of the low modulation frequency (L_Fmod) in Fig. 70 and the period T2 of the high modulation frequency (H_Fmod) in Fig. 71, when the low modulation frequency (L_Fmod) and the high modulation frequency (H_Fmod) are superimposed, a period T3 in which no light pulse is generated occurs periodically.

[0264] Fig. 74 shows the relationship between the ranging accuracy and ranging range for each ranging method. Fig. 74 shows three ranging methods: a first ranging method that uses only low-frequency signals to perform ranging, a second ranging method that uses only high-frequency signals to perform ranging, and a third ranging method that uses a superimposed signal of low-frequency and high-frequency signals to perform ranging.

[0265] When comparing the ranging accuracy of the three methods, the third ranging method uses a superimposed signal with a low-frequency signal, which means there are periods when no optical pulses are generated. This reduces the SNR (Signal-Noise Ratio) by half compared to the second ranging method, and the inherent high ranging accuracy may not be fully utilized. Therefore, the ranging accuracy is, in descending order, the second ranging method, the third ranging method, and the first ranging method. For example, ranging accuracy (ranging variation) is expressed by the following equation (10). In other words, the higher the modulation frequency (Fmod) and the higher the SNR, the smaller the ranging variation and the more accurate the accuracy.

[0266]

[0267] When comparing the ranging ranges of the three methods, the third ranging method uses the least common multiple of the ranging ranges of the low modulation frequency and the high modulation frequency, thereby enabling a long ranging range. Therefore, the ranging ranges are, in order from longest to shortest, the third ranging method, the first ranging method, and the second ranging method. The ranging device 1000 employs the third ranging method in dual mode and the second ranging method in single mode, and by appropriately switching between these modes, can, for example, operate in dual mode normally and in single mode when ranging accuracy is required. This makes it possible to achieve both a high level of ranging range and ranging accuracy.

[0268] <<Fifteenth Embodiment>> In the distance measuring device 1000, the distance measuring mode in the distance measuring module 1001 may be switched in accordance with an instruction from an external device connected to the host device 1002 via a network.

[0269] <System Configuration> Fig. 75 is a diagram showing an example configuration of a system including an edge camera equipped with a ranging device to which the present disclosure is applied and a cloud system. In Fig. 75, the edge camera group 1201 is made up of edge cameras 1211-1 to 1211-i (i is a natural number greater than or equal to 1). The edge cameras 1211-1 to 1211-i are installed in different locations.

[0270] The edge camera 1211-1, like the distance measuring device 1000 in Fig. 60, includes a distance measuring module 1001 and a host device 1002. The edge camera 1211-1 includes a communication device having a communication function using a mobile communication network such as 5G (5th Generation: 5th generation mobile communication system), and can exchange data with devices such as servers connected to a network such as the Internet. The edge cameras 1211-2 to 1211-i have the same configuration as the edge camera 1211-1.

[0271] The cloud system 1202 is composed of a cloud server and other components built in a cloud environment by a cloud service provider. In FIG. 75 , the cloud system 1202 includes an information processing device 1221 that performs various types of information processing and a storage device 1222 that records various types of data. For example, the cloud system 1202 provides services related to AI (artificial intelligence). The cloud system 1202 exchanges data with an AI application user terminal 1203, an AI application developer terminal 1204, or an AI model developer terminal 1205 via a network such as the Internet.

[0272] The AI ​​application user terminal 1203 is a terminal configured as a PC (Personal Computer), smartphone, etc., and is used by users who use AI applications. The AI ​​application developer terminal 1204 is a terminal configured as a PC, etc., and is used by developers who develop AI applications. The AI ​​model developer terminal 1205 is a terminal configured as a PC, etc., and is used by developers who develop AI models.

[0273] An AI app is an application equipped with AI. An AI model is a model (trained model) implemented in an AI app. An AI app is created by a developer using an AI app developer terminal 1204 and uploaded to the cloud system 1202 via a network. An AI model is created by a developer using an AI model developer terminal 1205 and uploaded to the cloud system 1202 via a network. When an AI app and an AI model implemented in the AI ​​app are uploaded from the AI ​​app developer terminal 1204 and the AI ​​model developer terminal 1205 to the cloud system 1202, the AI ​​app and the AI ​​model are associated with each other, recorded in the storage device 1222, and managed by the information processing device 1221. For example, an AI app is an application that performs image processing using ranging data. An AI model is a trained model generated by machine learning using training data (e.g., accumulated past ranging data, etc.). For example, a trained model can be generated using a neural network (NN). Hereinafter, an associated AI app and AI model will also be referred to as an "AI app / AI model." The AI ​​app and the AI ​​model may be developed by the same developer and on the same device.

[0274] When an AI application user uses an AI service provided by the cloud system 1202, the user operates the AI ​​application user terminal 1203 to select the desired AI application. The AI ​​application / AI model is then downloaded to the edge camera 1211-1 via the network and installed in the host device 1002. In the host device 1002, the processing unit 1133 performs analysis using the AI ​​application / AI model based on the ranging data output from the ranging module 1001, and uploads the image analysis results of the distance image to the cloud system 1202 via the network. The host device 1002 may also acquire information about the location of the edge camera 1211-1 and the imaging results of the surrounding area, and upload these to the cloud system 1202 via the network. For example, the location information can be acquired using a positioning system such as a GPS (Global Positioning System). The imaging results may use images captured by another camera included in the edge camera 1211-1, or distance images based on ranging data.

[0275] In the cloud system 1202, the information processing device 1221 generates a selection signal according to the image analysis results uploaded from the edge camera 1211-1 (host device 1002) and transmits it to the edge camera 1211-1 via the network. The information processing device 1221 may perform analysis using other information such as location information and imaging results in addition to the image analysis results, generate a selection signal according to the analysis results, and transmit it to the edge camera 1211-1. In the edge camera 1211-1, the selection signal transmitted from the information processing device 1221 in the cloud system 1202 is input to the imaging device 1012 of the ranging module 1001. In the imaging device 1012, the selection signal is input to the pixel light source driving unit 1141 (FIG. 62) of the photodetector element 1025.

[0276] In the pixel light source driving unit 1141 (FIG. 62), the signal selection unit 1154 selects the low-frequency signal generated by the low-frequency signal generation unit 1151 or the non-periodic signal generated by the non-periodic signal generation unit 1153 in accordance with a selection signal transmitted from the information processing device 1221. This allows the ranging module 1001 to perform ranging in dual mode or single mode. That is, in the ranging device 1000, the ranging mode in the ranging module 1001 is switched so as to select a mode suitable for the AI ​​application (AI application / AI model) selected on the cloud system 1202 side. Note that the edge camera 1211-1 or the host device 1002 may have a communication function for communicating with the cloud system 1202. At least a portion of the processing performed by the host device 1002 described above may be performed by the edge camera 1211-1.

[0277] <<16th Embodiment>> In the column processing unit 1073 shown in Fig. 37 , signal separation is performed by the signal separation unit 1092, but signal separation may also be performed in the pixel circuit 1031B of the pixel 1031. Fig. 76 shows the configuration of the column processing unit 1073 when signal separation is performed in the pixel circuit 1031B. In Fig. 76 , parts corresponding to those in Fig. 37 are assigned the same reference numerals, and their description will be omitted as appropriate. Compared to the column processing unit 1073 in Fig. 37 , the column processing unit 1073 in Fig. 76 does not include the signal separation unit 1092, and a signal corresponding to a low frequency from the readout unit 1091 is supplied to a low-frequency signal processing unit 1093, and a signal corresponding to a high frequency from the readout unit 1091 is supplied to a high-frequency signal processing unit 1094.

[0278] <Circuit Configuration and Operation of Pixel> Figure 77 is a diagram showing an example of the circuit configuration of a pixel 1031. As shown in Figure 77, of the laminated substrates 1051 and 1052, a photodiode PD, a charge discharging transistor OFG, a distribution transistor TG, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL are formed on the substrate 1051. Furthermore, a selection transistor FRQ, a subtractor 1301, a subtractor 1302, and an adder 1303 are formed on the substrate 1052.

[0279] In the substrate 1051, one of the source or drain of the distribution transistor TGA is electrically connected to the photodiode PD. The other of the source or drain of the distribution transistor TGA is electrically connected to one of the source or drain of each of the distribution transistors TGC and TGD. The other of the source or drain of the distribution transistor TGC is connected to a floating diffusion region FD_AC. The floating diffusion region FD_AC is electrically connected to the gate of the amplifier transistor AMP_AC. One of the source or drain of the amplifier transistor AMP_AC is electrically connected to one of the source or drain of the select transistor SEL. The other of the source or drain of the amplifier transistor AMP_AC is electrically connected to a power supply potential VDD. The other of the source or drain of the select transistor SEL is electrically connected to a subtractor 1301 formed in the substrate 1052.

[0280] The other of the source or drain of the distribution transistor TGD is connected to the floating diffusion region FD_AD. The floating diffusion region FD_AD is electrically connected to the gate of the amplification transistor AMP_AD. One of the source or drain of the amplification transistor AMP_AD is electrically connected to one of the source or drain of the selection transistor SEL. The other of the source or drain of the amplification transistor AMP_AD is electrically connected to the power supply potential VDD. The other of the source or drain of the selection transistor SEL is electrically connected to one of the source or drain of the selection transistors FRQ_H and FRQ_L formed on the substrate 1052.

[0281] In addition, in the substrate 1051, one of the source or drain of the distribution transistor TGB is electrically connected to the photodiode PD. The other of the source or drain of the distribution transistor TGB is electrically connected to one of the source or drain of the distribution transistors TGC and TGD. The other of the source or drain of the distribution transistor TGC is connected to the floating diffusion region FD_BC. The floating diffusion region FD_BC is electrically connected to the gate of the amplification transistor AMP_BC. One of the source or drain of the amplification transistor AMP_BC is electrically connected to one of the source or drain of the selection transistor SEL. The other of the source or drain of the amplification transistor AMP_BC is electrically connected to the power supply potential VDD. The other of the source or drain of the selection transistor SEL is electrically connected to one of the source or drain of the selection transistors FRQ_L and FRQ_H formed in the substrate 1052.

[0282] The other of the source or drain of the distribution transistor TGD is connected to the floating diffusion region FD_BD. The floating diffusion region FD_BD is electrically connected to the gate of the amplification transistor AMP_BD. One of the source or drain of the amplification transistor AMP_BD is electrically connected to one of the source or drain of the selection transistor SEL. The other of the source or drain of the amplification transistor AMP_BD is electrically connected to the power supply potential VDD. The other of the source or drain of the selection transistor SEL is electrically connected to a subtractor 1302 formed on the substrate 1052.

[0283] Of the four select transistors FRQ_H, FRQ_L, FRQ_L, FRQ_H arranged vertically in the drawing on the substrate 1052, one of the source / drain of the upper two select transistors FRQ_H, FRQ_L is electrically connected to the other of the source / drain of the select transistor SEL on the amplifying transistor AMP_AD side on the substrate 1051. Of the two upper select transistors FRQ_H, FRQ_L, the other of the source / drain of the select transistor FRQ_H is electrically connected to a subtractor 1301, and the other of the source / drain of the select transistor FRQ_L is electrically connected to a subtractor 1302.

[0284] Of the four select transistors FRQ_H, FRQ_L, FRQ_L, FRQ_H arranged vertically in the figure, one of the source / drain of the lower two select transistors FRQ_L, FRQ_H is electrically connected to the other of the source / drain of the select transistor SEL on the amplifier transistor AMP_BC side of the substrate 1051. Of the lower two select transistors FRQ_L, FRQ_H, the other of the source / drain of the select transistor FRQ_L is electrically connected to a subtractor 1301, and the other of the source / drain of the select transistor FRQ_H is electrically connected to a subtractor 1302.

[0285] Here, of the four selection transistors FRQ_H, FRQ_L, FRQ_L, FRQ_H, two selection transistors FRQ_L are turned on in response to the voltage applied to their gates when reading out a signal corresponding to a low frequency. That is, when reading out a signal corresponding to a low frequency, the two selection transistors FRQ_L are turned on, and the subtractor 1301 receives the signal Q from the amplification transistor AMP_AC. AC Along with this, the signal Q from the amplifier transistor AMP_BC BC The subtractor 1301 receives the signal Q AC and signal Q BC The subtraction result (Q AC -Q BC ) to the adder 1303. The subtractor 1302 also receives the signal Q from the amplifier transistor AMP_BD. BD Along with this, the signal Q from the amplifier transistor AMP_AD AD The subtractor 1302 receives the signal Q AD and signal Q BD The result of the subtraction (Q AD -Q BD ) to the adder 1303. The adder 1303 adds the subtraction result of the subtracter 1301 and the subtraction result of the subtracter 1302, and outputs the sum ((Q AC +Q AD )-(Q BC +Q BD)) is output to the readout unit 1091. This addition result corresponds to the above-mentioned equation (6), and signal separation is performed in the pixel circuit 1031B.

[0286] Furthermore, of the four selection transistors FRQ_H, FRQ_L, FRQ_L, and FRQ_H, two selection transistors FRQ_H are turned on in response to the voltage applied to their gates when reading out a signal corresponding to a high frequency. That is, when reading out a signal corresponding to a high frequency, the two selection transistors FRQ_H are turned on, and the subtractor 1301 receives the signal Q from the amplification transistor AMP_AC. AC Along with this, the signal Q from the amplifier transistor AMP_AD AD The subtractor 1301 receives the signal Q AC and signal Q AD The subtraction result (Q AC -Q AD ) to the adder 1303. The subtractor 1302 also receives the signal Q from the amplifier transistor AMP_BD. BD Along with this, the signal Q from the amplifier transistor AMP_BC BC The subtractor 1302 receives the signal Q BC and signal Q BD The subtraction result (Q BC -Q BD ) to the adder 1303. The adder 1303 adds the subtraction result of the subtracter 1301 and the subtraction result of the subtracter 1302, and outputs the sum ((Q AC +Q BC )-(Q AD +Q BD )) is output to the readout unit 1091. This addition result corresponds to the above-mentioned equation (7), and signal separation is performed in the pixel circuit 1031B.

[0287] Figure 78 is a timing chart including a signal readout operation in the equivalent circuit of the pixel 1031 shown in Figure 77. The pixel 1031 in Figure 77 is configured without a charge storage unit MEM, so the P-phase level is read out before the integration period and the D-phase level is read out after the integration period. Figure 78 shows operations during a P-phase readout period Ta, an integration period Tb, and a D-phase readout period Tc.

[0288] During the P-phase readout period Ta, the reset transistor RST is turned on to initialize the floating diffusion region FD, and the selection transistors SEL and FRQ_L are turned on to perform AD conversion of the P-phase level and obtain P-phase data (P). During the integration period Tb, the reflected light RL having a waveform in which a low modulation frequency and a high modulation frequency are superimposed is received by the photodiode PD, and the distribution transistors TGA and TGB operate at a drive voltage corresponding to the low modulation frequency, and the distribution transistors TGC and TGD operate at a drive voltage corresponding to the high modulation frequency.

[0289] During the D-phase readout period Tc, during the period when the selection transistor SEL is in the on state, first, the selection transistor FRQ_L is turned on, thereby AD-converting the D-phase level corresponding to the low frequency. Next, the selection transistor FRQ_H is turned on, thereby AD-converting the D-phase level corresponding to the high frequency. Then, CDS processing is performed to determine the difference between the P-phase data (P) and the D-phase data (D_LOW) as net pixel data, thereby acquiring a signal corresponding to the low frequency, which is output from the readout unit 1091 to the low-frequency signal processing unit 1093. Furthermore, CDS processing is performed to determine the difference between the P-phase data (P) and the D-phase data (D_HIGH) as net pixel data, thereby acquiring a signal corresponding to the high frequency, which is output from the readout unit 1091 to the high-frequency signal processing unit 1094.

[0290] <Effects> A distance measuring device to which the present disclosure is applied irradiates an object 2 with illumination light IL having a waveform in which multiple modulation frequencies (Fmod) are superimposed, separates a distance measurement signal obtained from reflected light RL when the illumination light IL is reflected by the object 2 for each modulation frequency (Fmod), and outputs distance measurement data (distance measurement information) that integrates distance measurement results obtained from the separated distance measurement signals. For example, when the structure of pixel 31 shown in FIG. 10 is adopted, a pair of taps (TG0, TG1) that distributes charge in the photodiode PD and two pairs of taps (TG2, TG3) that distribute the charge again after the charge is distributed by the first pair of taps are arranged. In this case, the light source waveform is a waveform in which two different modulation frequencies (L_Fmod, H_Fmod) are superimposed (formed by multiplication), and the drive voltage waveform is a voltage waveform with different modulation frequencies (L_Fmod, H_Fmod) applied to the pair of taps (TG0, TG1) and the two pairs of taps (TG2, TG3). As a result, in a ranging device to which the present disclosure is applied, even when two types of modulation frequencies (Fmod) are used to achieve both ranging range and ranging accuracy, ranging can be performed at different modulation frequencies (Fmod) in the same frame, thereby suppressing blurring that occurs when ranging a moving subject.

[0291] The effects obtained by a distance measuring device to which the present disclosure is applied will be described with reference to FIGS. 79 and 80 . For comparison, measurements using a conventional distance measuring device will be described with reference to FIG. 79 , and then measurements using a distance measuring device to which the present disclosure is applied will be described with reference to FIG. 80 . As shown in FIG. 79 , when distance measuring is performed using two modulation frequencies (Fmod), low frequency and high frequency, there is a discrepancy between the measurement start times of the low-frequency measurement period and the high-frequency measurement period, which may cause blurring when measuring the distance to a moving subject. In contrast, as shown in FIG. 80 , when distance measuring is performed using two modulation frequencies (Fmod), low frequency and high frequency, the measurement start times of the low-frequency and high-frequency frequencies are aligned (because low-frequency and high-frequency signals are acquired simultaneously), distance measuring can be performed at different modulation frequencies (Fmod) in the same frame. As a result, moving subject blurring can be suppressed.

[0292] <Modifications> In the distance measuring device 1 shown in FIG. 2 , the light receiving unit 12 is provided with a low-frequency signal generator 32 and a high-frequency signal generator 33. However, the control unit 14 may be provided with a low-frequency signal generator 32 and a high-frequency signal generator 33, and the generated low-frequency and high-frequency signals may be output to the light emitting unit 11 and the light receiving unit 12, respectively. In the distance measuring device 1 shown in FIG. 2 , the light receiving unit 12 is provided with a signal separator 34, a low-frequency signal processor 35, a high-frequency signal processor 36, and an integrated processor 37. However, at least one of these blocks may be provided as a signal processor. The light source 21 of the light emitting unit 11 may emit infrared light (IR) with a wavelength ranging from 780 nm to 1000 nm, for example, but is not limited to emitting such infrared light. The same applies to the distance measuring device 1 shown in FIG. 20 . In the above description, the first conductivity type is N-type and the second conductivity type is P-type, but the conductivity types of the components may be reversed.

[0293] In the present disclosure, illumination light IL having a waveform in which N modulation frequencies (N is a natural number greater than or equal to 2) are superimposed is used. However, the number of superimposed frequencies may or may not match the number of frequencies in the generated signal. For example, in the pixel light source drive unit 1072 of FIG. 36 , the modulation signal superimposing unit 1083 superimposes a low-frequency signal generated by the low-frequency signal generating unit 1081 and a high-frequency signal generated by the high-frequency signal generating unit 1082 to use two modulation frequencies, a low frequency and a high frequency. However, two or more modulation frequencies may be used. That is, if K frequency signal generating units are provided to generate K frequency signals (K types) each having a different modulation frequency, when the modulation signal superimposing unit 1083 generates a superimposed signal in which N frequencies (N types) are superimposed, the relationship N≧K>1 may be satisfied. In this case, the K frequency signals may be generated by a single frequency signal generating unit. 37 , the signal separation unit 1092 may separate the signal from the readout unit 1091 into K separated signals corresponding to the K modulation frequencies, and K frequency signal processing units may be provided to acquire K ranging results, which are measurement results at the K modulation frequencies, based on the K separated signals from the signal separation unit 1092. In this case, one frequency signal processing unit may acquire K ranging results based on the K separated signals. The integration processing unit 1095 may integrate the K ranging results from the frequency signal processing units to generate distance information.

[0294] <Other Application Examples> While the above description illustrates application examples of the present disclosure, such as volume detection of an object moving on a conveyor belt, gesture recognition inside a vehicle, and installation in a surveillance camera, the application examples are not limited to these. For example, a distance measuring device to which the present disclosure is applied may be installed in a game console to recognize a user's gestures or hand signals, or in a security camera to measure the face shape and body shape, including height, of a criminal. Alternatively, a distance measuring device to which the present disclosure is applied may be installed in an autonomous robot or vehicle to measure the shapes of surrounding objects and obstacles, or may be applied to a device equipped with a camera for virtual reality (VR) or augmented reality (AR) to measure the shapes of surrounding objects and moving bodies.

[0295] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the present disclosure. Each embodiment may be implemented alone, or two or more embodiments may be combined. For example, the pixel structure of the twelfth embodiment ( FIG. 40 ) may be configured as a vertical transistor, similar to the pixel structure of the third embodiment ( FIG. 13 ). The pixel structure of the twelfth embodiment ( FIG. 40 ) may be configured as a CAPD structure, similar to the pixel structure of the fourth embodiment ( FIG. 15 ), or as a CDTI structure, similar to the pixel structure of the sixth embodiment ( FIG. 18 ). The effects described herein are merely examples and are not limiting, and other effects may also be present.

[0296] The present disclosure can also be configured as follows.

[0297] (1) A distance measuring device comprising: an emitter that emits illumination light; and a light receiver that receives light reflected from an object, wherein the illumination light has a waveform formed by superimposing N (N is a natural number of 2 or more) modulation frequencies; and the light receiver separates ranging signals obtained from the reflected light for each modulation frequency and outputs distance information that integrates ranging results obtained from the separated ranging signals. (2) The distance measuring device described in (1), wherein the illumination light has a waveform formed by multiplying N different modulation frequencies. (3) The light receiver includes a pixel array unit in which a plurality of pixels, each having a photoelectric conversion unit, are two-dimensionally arranged, and applies voltages of drive voltage waveforms with different modulation frequencies to the N pixels. (4) The distance measuring device according to (2), wherein the light receiving unit includes a pixel array unit in which a plurality of pixels each having a photoelectric conversion unit are arranged two-dimensionally, and voltages of drive voltage waveforms of N modulation frequencies are applied to one pixel. (5) The distance measuring device according to (4), wherein the pixel has taps for distributing electric charges generated in the photoelectric conversion unit, and each tap is formed of a MOS transistor. (6) The distance measuring device according to (4), wherein m pairs of taps are formed within one pixel. The distance measuring device according to (5). (7) The distance measuring device according to (6), in which N=2 and three pairs of taps are formed, and one pair of taps out of the three pairs of taps is arranged inside the other two pairs of taps when the center of the pixel is used as a reference. (8) The distance measuring device according to (7), in which the pair of taps is formed by a vertical gate. (9) The distance measuring device according to (7) or (8), in which the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, and a voltage having a drive voltage waveform of the first modulation frequency is applied to the pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps. (10) The distance measuring device according to (4), in which the pixel has taps that distribute electric charges generated in the photoelectric conversion unit, and each tap is formed of an impurity. (11) The distance measuring device according to (4), in which m pairs of taps are formed within one pixel. The distance measuring device according to (10). (12) The distance measuring device according to (11), in which N=2 and three pairs of taps are formed. (13) The distance measuring device according to (12), in which one pair of taps out of the three pairs of taps is arranged inside the other two pairs of taps when the center of the pixel is used as a reference. (14) The distance measuring device according to (13), in which the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, and a voltage having a drive voltage waveform of the first modulation frequency is applied to the one pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps. (15) The distance measuring device according to (12), in which one pair of taps out of the three pairs of taps is arranged outside the other two pairs of taps when the center of the pixel is used as a reference. (16) The distance measuring device according to (15), wherein the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, and a voltage having a drive voltage waveform of the first modulation frequency is applied to the pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps. (17) The distance measuring device according to (4), wherein the pixel has taps that distribute electric charges generated in the photoelectric conversion unit, and each tap is formed by covering a groove formed in a substrate with an insulator and burying a conductor therein. (18) The distance measuring device according to (4), wherein each tap has two taps within one pixel. N(19) The distance measuring device according to (18), in which N=2 and four taps are formed, the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, a voltage having a drive voltage waveform of the first modulation frequency is applied to two taps arranged opposite each other in a first direction, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the remaining two taps arranged opposite each other in a second direction that is perpendicular to the first direction. (20) The distance measuring device according to (2), in which the light receiving unit includes a pixel array unit in which a plurality of pixels each having a photoelectric conversion unit are arranged, and a voltage having a drive voltage waveform formed by multiplying N different modulation frequencies is applied to the pixels. (21) The pixel has a tap that distributes electric charges generated in the photoelectric conversion unit, and each tap has two taps within one pixel. NThe distance measuring device according to (20), wherein each tap is formed of a MOS transistor. (22) The distance measuring device according to (21), wherein each tap is formed of an impurity. (23) The distance measuring device according to (21), wherein each tap is formed of a photogate. (24) The distance measuring device according to (21), wherein each tap is formed of a photogate. (25) The distance measuring device according to (1) or (2), wherein the irradiated light has a waveform in which N modulation frequencies are superimposed, and each contains light corresponding to different polarization information. (26) The distance measuring device according to (25), wherein the light receiving unit includes a pixel array unit in which a plurality of pixels, each having a photoelectric conversion unit, are two-dimensionally arranged, and a polarizer corresponding to the polarization information is installed on the irradiation surface side of the pixel. (27) The distance measuring device according to (1), further comprising a driver that generates signals for driving a light source included in the light-emitting unit and pixels arranged two-dimensionally in a pixel array unit included in the light-receiving unit, wherein the driver has a signal generator that generates K (N≧K>1) frequency signals, each having a different modulation frequency. (28) The distance measuring device according to (27), wherein the driver further has a superimposing unit that generates a superimposed signal by superimposing the K frequency signals, wherein the light source is driven based on the superimposed signal. (29) The distance measuring device according to (28), wherein the pixel is driven based on the superimposed signal. (30) The distance measuring device according to (27), wherein the pixel is driven based on the K frequency signals. (31) The distance measuring device according to any of (27) to (30), wherein the K-th modulation frequency of the K-th frequency signal is higher than the K-1-th modulation frequency of the K-1-th frequency signal. (32) The distance measuring device according to (31), wherein the Kth modulation frequency is an integer multiple of the K-1th modulation frequency. (33) The distance measuring device according to (27), wherein the K frequency signals each have information relating to a different phase. (34) The distance measuring device according to (27) or (28), wherein the light receiving unit includes a pixel array unit in which a plurality of pixels, each having a photoelectric conversion unit, are arranged two-dimensionally, and each pixel has a pixel circuit that performs analog signal processing.(35) The ranging device according to (34), wherein the pixel circuit separates a signal obtained from the charge generated in the photoelectric conversion unit into K separated signals corresponding to K modulation frequencies. (36) The ranging device according to (34), wherein the light receiving unit includes a column processing unit that processes the signal of each pixel for each column to generate the distance information, and the column processing unit includes a readout unit that performs AD conversion and CDS processing on the signal of each pixel. (37) The ranging device according to (36), wherein the column processing unit further includes a signal separation unit that separates the signal from the readout unit into K separated signals corresponding to the K modulation frequencies. (38) The ranging device according to (37), wherein the column processing unit further includes a signal processing unit that acquires K ranging results, which are ranging results at the K modulation frequencies, based on the K separated signals from the signal separation unit. (39) The ranging device according to (38), wherein the column processing unit further includes an integration processing unit that integrates the K ranging results from the signal processing unit to generate the distance information. (40) The distance measuring device according to (28), wherein the driver further includes a non-periodic signal generator that generates a non-periodic signal without information related to frequency. (41) The distance measuring device according to (40), wherein the K-1th modulation frequency of the K-1th frequency signal is lower than the Kth modulation frequency of the Kth frequency signal, and wherein the driver further includes a signal selector that selects the K-1st modulation frequency or the non-periodic signal based on a selection signal that selects a signal for driving the light source and the pixel. (42) The distance measuring device according to (41), further including a host device that controls the light receiving unit, wherein the host device includes an image processing unit that performs image processing to detect a specific object from a distance image that corresponds to the distance information output from the light receiving unit. (43) The distance measuring device according to (42), wherein the image processing unit outputs the selection signal that corresponds to the result of the image processing to the light receiving unit. (44) The distance measuring device according to (41), further comprising a host device that controls the light receiving unit, wherein the host device is connected via a network to an information processing device that constitutes a system that provides AI-related services.(45) The distance measuring device according to (44), in which an application and a trained model managed by the information processing device are installed in the host device. (46) The distance measuring device according to (45), in which the host device performs an analysis using the application and the trained model based on the distance information output from the light receiving unit and transmits the analysis result to the information processing device. (47) The distance measuring device according to (46), in which the signal selection unit selects the K-1 modulation frequency or the non-periodic signal based on the analysis result transmitted from the information processing device. (48) A distance measuring method, comprising: a distance measuring device emitting irradiation light having a waveform in which N modulation frequencies (N is a natural number of 2 or more) are superimposed, receiving light reflected from an object from which the irradiation light is reflected, and separating distance measurement signals obtained from the reflected light for each modulation frequency, and outputting distance information integrating distance measurement results obtained from the separated distance measurement signals.

[0298] REFERENCE SIGNS LIST 1 Range finding device, 11 Light emitting unit, 12, 12A Light receiving unit, 13 Distance image output unit, 14 Control unit, 21 Light source, 22 Light emission control unit, 23 Modulation signal superimposing unit, 31, 31-1, 31-2 Pixel, 31A Photoelectric conversion unit, 31B Pixel circuit, 32 Low frequency signal generating unit, 33 High frequency signal generating unit, 34 Signal separation unit, 35 Low frequency signal processing unit, 36 High frequency signal processing unit, 37 Integrated processing unit, 38 Modulation signal superimposing unit, 51 Substrate, 52 Substrate, 61 Pixel array unit, 71 Vertical control unit, 101, 111 Gate electrode, 201, 202, 206, 211, 212, 216 Gate electrode, 301, 302, 306, 311, 312, 316 Gate electrode, 411 N-type semiconductor region, 412 P-type semiconductor region, 413 P-type semiconductor region, 414 N-type semiconductor region, 451 N-type semiconductor region, 452 N-type semiconductor region, 453 Insulator, 454 Conductor, 501, 511, 521, 531 Gate electrode, 601, 611, 621, 631 Gate electrode, 701, 711, 721, 731, 741, 751, 761, 771 Gate electrode, 811-1, 811-2 Polarizer, 1000 Range finding device, 1001 Range finding module, 1002 Host device, 1011 Light source device, 1012 Image capturing device, 1021 Light-emitting control unit, 1022 light-emitting unit, 1023 lens, 1024 lens, 1025 photodetector element, 1031 pixel, 1031A photoelectric conversion unit, 1031B pixel circuit, 1051 substrate, 1052 substrate, 1061 pixel array unit, 1071 vertical control unit, 1072 pixel light source drive unit, 1073 column processing unit, 1074 horizontal drive unit, 1075 system control unit, 1081 low-frequency signal generation unit, 1082 high-frequency signal generation unit, 1083 modulation signal superposition unit, 1091 readout unit, 1092 signal separation unit, 1093 low-frequency signal processing unit, 1094 high-frequency signal processing unit, 1095 integration processing unit, 1096 ranging data transmission unit, 1111 Pixel light source driving unit, 1121 low frequency signal generating unit, 1122 high frequency signal generating unit, 1123 modulation signal superimposing unit, 1131 image processing unit, 1132 memory, 1141 pixel light source driving unit, 1151 low frequency signal generating unit, 1152 high frequency signal generating unit1153 Non-periodic signal generation unit, 1154 Signal selection unit, 1155 Modulation signal superposition unit, 1201 Edge camera group, 1202 Cloud system, 1203 AI application user terminal, 1204 AI application developer terminal, 1205 AI model developer terminal, 1211 Edge camera, 1221 Information processing device, 1222 Storage device, PD Photodiode, TG0, TG1, TG2, TG3, TG4, TG5, TG6, TG7 Distribution transistor, VG0, VG1 Distribution transistor, P0, P1, P2, P3 Semiconductor area, N0, N1, N2, N3 Semiconductor area, CDTI-0, CDTI-1, CDTI-2, CDTI-3 Voltage application unit, CDTI-A, CDTI-B, CDTI-C, CDTI-D Voltage application unit, FD-A, FD-B, FD-C, FD-D: charge detection section, PG0, PG1, PG2, PG3: photogate, TGA, TGB, TGC, TGD: distribution transistor, FRQ_L, FRQ_H: selection transistor,

Claims

1. A distance measuring device comprising: an emitter for emitting irradiated light; and a light receiver for receiving light reflected from an object, wherein the irradiated light has a waveform formed by superimposing N modulation frequencies (N is a natural number of 2 or more), and the light receiver separates distance measuring signals obtained from the reflected light for each modulation frequency, and outputs distance information that integrates distance measuring results obtained from the separated distance measuring signals.

2. The distance measuring device according to claim 1, wherein the irradiated light has a waveform formed by multiplying N different modulation frequencies.

3. The distance measuring device according to claim 2, wherein the light receiving section comprises a pixel array section in which a plurality of pixels, each having a photoelectric conversion section, are arranged two-dimensionally, and voltages having drive voltage waveforms with different modulation frequencies are applied to the N pixels.

4. The distance measuring device according to claim 2, wherein the light receiving section comprises a pixel array section in which a plurality of pixels, each having a photoelectric conversion section, are arranged two-dimensionally, and voltages having driving voltage waveforms of N modulation frequencies are applied to each pixel.

5. The distance measuring device according to claim 4, wherein the pixel has a tap for distributing the electric charge generated in the photoelectric conversion portion, and each tap is formed of a MOS transistor.

6. Each tap is formed into m pairs within one pixel.

6. A distance measuring device according to claim 5.

7. The distance measuring device according to claim 6, wherein N=2, three pairs of taps are formed, and one pair of taps of the three pairs of taps is positioned inward relative to the other two pairs of taps when the center of the pixel is used as a reference.

8. The distance measuring device according to claim 7, wherein the pair of taps are formed by a vertical gate.

9. A distance measuring device as described in claim 7, wherein the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, a voltage having a drive voltage waveform of the first modulation frequency is applied to the pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps.

10. The distance measuring device according to claim 4, wherein the pixel has a tap for distributing the electric charge generated in the photoelectric conversion portion, and each tap is formed of an impurity.

11. Each tap is formed into m pairs within one pixel. The distance measuring device according to claim 10.

12. The distance measuring device according to claim 11, wherein N=2 and three pairs of taps are formed.

13. The distance measuring device according to claim 12, wherein, of the three pairs of taps, one pair of taps is disposed on the inside of the other two pairs of taps when the center of the pixel is used as a reference.

14. A distance measuring device as described in claim 13, wherein the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, a voltage having a drive voltage waveform of the first modulation frequency is applied to the pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps.

15. The distance measuring device according to claim 12, wherein, of the three pairs of taps, one pair of taps is disposed on the outer side with respect to the other two pairs of taps when the center of the pixel is used as a reference.

16. A distance measuring device as described in claim 15, wherein the two modulation frequencies include a first modulation frequency and a second modulation frequency that is higher than the first modulation frequency, a voltage having a drive voltage waveform of the first modulation frequency is applied to the pair of taps, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the other two pairs of taps.

17. A distance measuring device as described in claim 4, wherein the pixel has a tap for distributing the electric charge generated in the photoelectric conversion portion, and each tap is formed by covering a groove formed in a substrate with an insulator and embedding a conductor therein.

18. Each tap has 2 pixels in one pixel. N The distance measuring device according to claim 17 , 19. A distance measuring device as described in claim 18, wherein N=2, four taps are formed, the two modulation frequencies include a first modulation frequency and a second modulation frequency which is a higher frequency than the first modulation frequency, a voltage having a drive voltage waveform of the first modulation frequency is applied to two taps arranged opposite each other in a first direction, and a voltage having a drive voltage waveform of the second modulation frequency is applied to the remaining two taps arranged opposite each other in a second direction which is perpendicular to the first direction.

20. The distance measuring device according to claim 2, wherein the light receiving section includes a pixel array section in which a plurality of pixels, each having a photoelectric conversion section, are arranged, and a voltage having a drive voltage waveform formed by multiplying N different modulation frequencies is applied to the pixels.

21. The pixel has a tap for distributing the electric charge generated in the photoelectric conversion unit, and each tap has two N The distance measuring device according to claim 20 , 22. The distance measuring device according to claim 21, wherein each tap is formed of a MOS transistor.

23. The distance measuring device according to claim 21, wherein each tap is formed from an impurity.

24. The distance measuring device according to claim 21, wherein each tap is formed by a photogate.

25. The distance measuring device according to claim 1, wherein the irradiated light has a waveform in which N modulation frequencies are superimposed, and each of the modulation frequencies contains light corresponding to different polarization information.

26. A distance measuring device as described in claim 25, wherein the light receiving unit has a pixel array section in which a plurality of pixels, each having a photoelectric conversion unit, are arranged two-dimensionally, and a polarizer corresponding to the polarization information is installed on the irradiation surface side of the pixel.

27. A distance measuring device as described in claim 1, further comprising a driver unit that generates signals for driving a light source provided in the light emitting unit and pixels arranged two-dimensionally in a pixel array provided in the light receiving unit, the driver unit having a signal generator unit that generates K frequency signals (N≧K>1) each having a different modulation frequency.

28. The distance measuring device according to claim 27, wherein the driving section further includes a superimposing section that superimposes the K frequency signals to generate a superimposed signal, and the light source is driven based on the superimposed signal.

29. The distance measuring device according to claim 28, wherein the pixel is driven based on the superimposed signal.

30. The distance measuring device according to claim 27, wherein the pixels are driven based on the K frequency signals.

31. The distance measuring device according to claim 27, wherein the Kth modulation frequency of the Kth frequency signal is higher than the K-1th modulation frequency of the K-1th frequency signal.

32. The distance measuring device according to claim 31, wherein the Kth modulation frequency is an integer multiple of the K-1th modulation frequency.

33. The distance measuring device according to claim 27, wherein the K frequency signals each have different phase information.

34. The distance measuring device according to claim 27, wherein the light receiving section comprises a pixel array section in which a plurality of pixels, each having a photoelectric conversion section, are arranged two-dimensionally, and each pixel has a pixel circuit that performs analog signal processing.

35. The distance measuring device according to claim 34, wherein the pixel circuit separates a signal obtained from the charge generated in the photoelectric conversion section into K separate signals corresponding to K modulation frequencies.

36. The distance measuring device according to claim 34, wherein the light receiving unit includes a column processing unit that processes the signals of each pixel for each column to generate the distance information, and the column processing unit has a readout unit that performs AD conversion and CDS processing on the signals of each pixel.

37. The distance measuring device according to claim 36, wherein the column processing section further comprises a signal separation section that separates the signal from the readout section into K separate signals corresponding to the K modulation frequencies.

38. The distance measuring device according to claim 37, wherein the column processing unit further includes a signal processing unit that acquires K distance measurement results, which are distance measurement results using the K modulation frequencies, based on the K separated signals from the signal separation unit.

39. The distance measuring device according to claim 38, wherein the column processing unit further includes an integration processing unit that integrates the K distance measurement results from the signal processing unit to generate the distance information.

40. The distance measuring device according to claim 28, wherein the driving section further comprises a non-periodic signal generating section that generates a non-periodic signal having no information regarding frequency.

41. A distance measuring device as described in claim 40, wherein the K-1th modulation frequency of the K-1th frequency signal is a frequency lower than the Kth modulation frequency of the Kth frequency signal, and the driving unit further has a signal selection unit that selects the K-1st modulation frequency or the non-periodic signal based on a selection signal that selects a signal for driving the light source and the pixel.

42. A distance measuring device as described in claim 41, further comprising a host device for controlling the light receiving unit, the host device having an image processing unit which performs image processing to detect a specific object from a distance image corresponding to the distance information output from the light receiving unit.

43. The distance measuring device according to claim 42, wherein the image processing section outputs the selection signal according to the result of the image processing to the light receiving section.

44. The distance measuring device according to claim 41, further comprising a host device that controls the light receiving unit, the host device being connected via a network to an information processing device that constitutes a system that provides AI-related services.

45. The distance measuring device according to claim 44, wherein an application and a trained model managed by the information processing device are installed on the host device.

46. ​​The distance measuring device according to claim 45, wherein the host device performs an analysis using the application and the trained model based on the distance information output from the light receiving unit, and transmits the analysis results to the information processing device.

47. A distance measuring device as described in claim 46, wherein the signal selection section selects the K-1 modulation frequency or the non-periodic signal based on the analysis result transmitted from the information processing device.

48. A distance measuring method comprising: a distance measuring device emitting illumination light having a waveform shape in which N (N is a natural number equal to or greater than 2) modulation frequencies are superimposed; receiving light reflected from an object from the illumination light; separating distance measuring signals obtained from the reflected light for each modulation frequency; and outputting distance information that integrates distance measuring results obtained from the separated distance measuring signals.

Images