Distance measuring device, distance measuring system, mobile object, and equipment

Abstract

To provide a distance measuring device that realizes distance measurement using a ToF method and a phase difference method and improves distance measurement accuracy.SOLUTION: A distance measuring device includes a light emitting unit that irradiates an object with light, a plurality of pixels, a microlens arranged on the plurality of pixels and shared by the plurality of pixels, a first distance measuring unit that obtains the distance to the object on the basis of a time it takes for the light irradiated with by the light emitting unit to be reflected, and a second distance measuring unit that obtains the distance to the object on the basis of the difference in output of each of the plurality of pixels that receive the reflected light.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a distance measuring device, a distance measuring system, a moving body, and an instrument. [Background technology]

[0002] Known methods for measuring the distance to an object include a ToF (Time of Flight) method and a phase difference method using a DAF (Dual Pixel Auto Focus) function. The ToF method is a method for acquiring the distance to an object based on the time it takes for irradiated light to be reflected by the object and return. The phase difference method is a method for acquiring the distance to an object based on the phase difference between signals output from two pixels included in an imaging element having a DAF function. Patent Document 1 discloses a technique for measuring distance using a direct ToF method (dToF method) and an indirect ToF method (iToF method). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-063415 Summary of the Invention [Problem to be solved by the invention]

[0004] However, because the accuracy of distance measurement using the ToF method depends on the reflectance of the object being measured, it may be difficult to obtain sufficient distance measurement accuracy when measuring distance using the dToF method and the iToF method.

[0005] Therefore, an object of the present invention is to provide a distance measuring device that realizes distance measurement using the ToF method and the phase difference method and improves distance measurement accuracy. [Means for solving the problem]

[0006] A first aspect of the present invention is a distance measuring device characterized by having an light emitting unit that irradiates light onto an object, a plurality of pixels, a microlens arranged on the plurality of pixels and shared by the plurality of pixels, a first distance measuring unit that obtains the distance to the object based on the time it takes for the light irradiated by the light emitting unit to be reflected and a second distance measuring unit that obtains the distance to the object based on the difference in output of each of the plurality of pixels that receive the reflected light.

[0007] A second aspect of the present invention is a distance measuring system comprising the distance measuring device described above and a signal processing unit that processes a signal output by the distance measuring device.

[0008] A third aspect of the present invention is a moving body equipped with the above-mentioned ranging device, characterized in that the moving body has a control unit that controls the movement of the moving body using information on the distance to the object acquired by the ranging device.

[0009] A fourth aspect of the present invention is an apparatus characterized by having the above-mentioned ranging device, an optical device corresponding to the ranging device, a control device controlling the ranging device, a processing device processing a signal output from the ranging device, a display device displaying information obtained by the ranging device, a storage device storing information obtained by the ranging device, and a mechanical device operating based on the information obtained by the ranging device. Effect of the Invention

[0010] According to the present invention, distance measurement using the ToF method and the phase difference method can be realized, and distance measurement accuracy can be improved. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is a schematic diagram of a distance measuring device. [Diagram 2] FIG. 2 is a schematic diagram of a pixel substrate of the distance measuring device. [Diagram 3] FIG. 2 is a schematic diagram of a circuit board of the distance measuring device. [Figure 4]2 is a configuration example of a pixel circuit of a distance measuring device. [Diagram 5] FIG. 2 is a schematic diagram showing the driving of a pixel circuit of the distance measuring device. [Figure 6] FIG. 2 is a plan view of a pixel of the distance measuring device. [Figure 7] 2 is a cross-sectional view of a pixel of the distance measuring device taken along line AA'. FIG. [Figure 8] FIG. 5 is a cross-sectional view of a pixel of the distance measuring device taken along the line BB'. [Figure 9] 4 is a cross-sectional view of a pixel of the distance measuring device taken along the line CC'. FIG. [Figure 10] FIG. 2 is a functional block diagram of the distance measuring device. [Figure 11] FIG. 11 is a diagram illustrating an example of a histogram representing the intensity of reflected light. [Figure 12] FIG. 1 is a diagram illustrating distance measurement using a phase difference method. [Figure 13] 11 is a flowchart illustrating a distance measurement process. [Figure 14] FIG. 1 is a diagram illustrating timing jitter. [Figure 15] FIG. 13 is a diagram illustrating a distance measuring system according to a second embodiment. [Figure 16] FIG. 13 is a diagram illustrating a distance measuring system and a moving object according to a third embodiment. [Figure 17] FIG. 13 is a diagram illustrating a range image sensor according to a fourth embodiment. [Figure 18] FIG. 13 is a diagram illustrating an endoscopic surgery system according to a fifth embodiment. [Figure 19] FIG. 13 is a diagram illustrating smart glasses according to a sixth embodiment. [Figure 20] FIG. 13 is a diagram illustrating an electronic device according to a seventh embodiment. [Figure 21] FIG. 13 is a diagram illustrating an X-ray CT apparatus according to an eighth embodiment. [Figure 22] FIG. 13 is a diagram illustrating an imaging system according to a ninth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) will be used as necessary. The use of these terms is for the purpose of facilitating understanding of the embodiment with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms.

[0014] In this specification, the plan view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer. Also, the cross-sectional view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer. Note that, when the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0015] The semiconductor layer has a first surface and a second surface, which is the surface opposite to the first surface and into which light is incident. In this specification, the depth direction is the direction from the first surface to the second surface of the semiconductor layer on which the avalanche photodiode (APD) is arranged. Hereinafter, the "first surface" may be referred to as the "front surface" and the "second surface" may be referred to as the "rear surface." The "depth" of a point or region in a semiconductor layer refers to the distance from the first surface (front surface) of that point or region. A point (or region) Z1 whose distance (depth) from the first surface is d1 and a point (or region) Z2 whose distance (depth) from the first surface is d2 are referred to as the "depth" of the first surface. When there is a point (or area) Z2 whose distance (depth) from the first surface is d3 and d1>d2 holds, it can be expressed as "Z1 is deeper than Z2" or "Z2 is shallower than Z1". When there is a point (or area) Z3 whose distance (depth) from the first surface is d3 and d1>d3>d2 holds, it can be expressed as "Z3 is at a depth between Z1 and Z2" or "Z3 is between Z1 and Z2 in the depth direction".

[0016] In the following description, the anode of the avalanche photodiode (APD) is set to a fixed potential, and a signal is taken from the cathode side. Therefore, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is an N-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is a P-type semiconductor region. The present invention is also valid when the cathode of the APD is set to a fixed potential, and a signal is taken from the anode side. In this case, the first conductive type semiconductor region having charges of the same polarity as the signal charge as the majority carrier is a P-type semiconductor region, and the second conductive type semiconductor region having charges of a different polarity from the signal charge as the majority carrier is an N-type semiconductor region. In the following, a case where one node of the APD is set to a fixed potential will be described, but the potentials of both nodes may fluctuate.

[0017] In this specification, when the term "impurity concentration" is used simply, it means the net impurity concentration minus the amount compensated by the impurity of the opposite conductivity type. In other words, "impurity concentration" refers to the NET doping concentration. A region where the P-type doped impurity concentration is higher than the N-type doped impurity concentration is a P-type semiconductor region. Conversely, a region where the N-type doped impurity concentration is higher than the P-type doped impurity concentration is an N-type semiconductor region.

[0018] In addition, in the following embodiments, the connection between elements of a circuit may be described. In this case, even if another element is interposed between the elements of interest, the elements of interest are treated as being electrically connected to each other unless otherwise specified. For example, assume that an element A is connected to one node of a capacitive element C having multiple nodes, and an element B is connected to the other node. Even in such a case, the elements A and B are treated as being electrically connected to each other unless otherwise specified. In addition, when elements are connected to each other without other elements being interposed, they may be described as being directly connected. In the above example, when no other elements are provided between the element A and the capacitive element C, it can be said that the element A and the capacitive element C are directly connected.

[0019] Metallic members such as wiring and pads described in this specification may be composed of a single metal element or may be a mixture (alloy). For example, wiring described as copper wiring may be composed of a single copper element or may be composed mainly of copper and further contain other components. Also, for example, pads connected to external terminals may be composed of a single aluminum element or may be composed mainly of aluminum and further contain other components. The copper wiring and aluminum pads shown here are examples and can be changed to various metals.

[0020] Moreover, the wiring and pads shown here are one example of metal members used in a photoelectric conversion device, and the present invention can also be applied to other metal members.

[0021] In each of the embodiments described below, a distance measuring device (a device for measuring distance using focus detection or ToF (Time Of Flight)) will be mainly described as an example of a photoelectric conversion device. However, each of the embodiments is not limited to a distance measuring device, and can be applied to other examples of photoelectric conversion devices, such as an imaging device and a photometric device (a device for measuring the amount of incident light, etc.).

[0022] A configuration common to each embodiment of a photoelectric conversion device and a driving method thereof according to the present invention will be described with reference to FIGS. 1 to 5. FIG.

[0023] FIG. 1 is a diagram showing a configuration of a photoelectric conversion device 100 according to an embodiment of the present invention. In the following, a case where the photoelectric conversion device 100 is a stacked type photoelectric conversion device will be described as an example. That is, a photoelectric conversion device configured by stacking and electrically connecting two substrates, a sensor substrate 11 and a circuit substrate 21, will be described as an example. However, the photoelectric conversion device is not limited to this. For example, the photoelectric conversion device may be a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate 21 are arranged on a common semiconductor layer, as described below. In the following, a photoelectric conversion device in which the configuration included in the sensor substrate 11 and the configuration included in the circuit substrate 21 are arranged on a common semiconductor layer will also be referred to as a non-stacked photoelectric conversion device.

[0024] The sensor substrate 11 has a first semiconductor layer having photoelectric conversion elements 102 described later, and a first wiring structure. The circuit substrate 21 has a second semiconductor layer having circuits such as a signal processing unit 103 described later, and a second wiring structure. The photoelectric conversion device 100 is configured by laminating the second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer in this order.

[0025] 1 illustrates a back-illuminated photoelectric conversion device in which light is incident from the second surface (rear surface) and a circuit board 21 is disposed on the first surface (front surface) that is the surface opposite the second surface. In the case of a non-stacked photoelectric conversion device, the surface on which the transistors of the signal processing circuit are disposed is referred to as the first surface. In the case of a front-illuminated photoelectric conversion device, the front surface is the second surface (light incident surface) and the back surface is the first surface.

[0026] In the following, the sensor substrate 11 and the circuit substrate 21 will be described as diced chips, but are not limited to chips. For example, each substrate may be a wafer. Also, each substrate may be stacked in a wafer state and then diced, or each chip may be stacked and bonded after being chipped.

[0027] A pixel region 12 is disposed on the sensor substrate 11, and a circuit region 22 that processes signals detected in the pixel region 12 is disposed on the circuit substrate 21.

[0028] 2 is a layout diagram of the sensor substrate 11. Pixels 101, each having a photoelectric conversion element 102 including an avalanche photodiode (APD), are arranged two-dimensionally to form a pixel region 12.

[0029] The pixel 101 is typically a pixel for forming an image, but when used for TOF (Time of Flight), it does not necessarily have to form an image. That is, the pixel 101 may be a pixel for measuring the time when light arrives and the amount of light.

[0030] 3 is a configuration diagram of the circuit board 21. It has a signal processing unit 103 that processes electric charges photoelectrically converted by the photoelectric conversion element 102 in FIG. 2, a readout circuit 112, a control pulse generating unit 115, a horizontal scanning circuit unit 111, signal lines 113, a vertical scanning circuit unit 110, drive lines 116, and an output circuit 114.

[0031] The photoelectric conversion element 102 in FIG. 2 and the signal processing unit 103 in FIG. 3 are electrically connected via connection wiring provided for each pixel.

[0032] The vertical scanning circuit section 110 receives a control pulse supplied from a control pulse generating section 115, and supplies the control pulse to each pixel via a driving line 116. The vertical scanning circuit section 110 uses logic circuits such as a shift register and an address decoder.

[0033] The control pulse generating unit 115 is a signal generating unit that generates a control signal P_CLK for the switch, which will be described later. The signal generating unit 215 has a pulse signal for controlling the switch, as described later. For example, as shown in FIG. 4(A), the signal generating unit 215 may generate a common control signal P_CLK for a plurality of pixels in the pixel region, or as shown in FIG. 4(B), the signal generating unit 215 may generate a control signal P_CLK for each pixel. When the pulse signal P_CLK is generated in common, at least one of the period, the number of pulses, and the pulse width of the signal P_EXP for controlling the exposure period is generated in common in correspondence with the exposure period. When the control signal P_CLK is controlled for each pixel, a signal can be generated using both the input signal P_CLK_IN output from the control pulse generating unit 115 and the signal P_EXP for controlling the exposure period. For example, the control pulse generating unit 115 preferably has a frequency dividing circuit. This allows simple control and reduces an increase in the number of elements.

[0034] The signal output from the photoelectric conversion element 102 of each pixel is processed by a signal processing unit 103. The signal processing unit 103 is provided with a counter, a memory, etc., and digital values ​​are held in the memory.

[0035] The horizontal scanning circuit unit 111 inputs a control pulse for sequentially selecting each column to the signal processing unit 103 in order to read out the signal from the memory of each pixel in which the digital signal is held.

[0036] A signal is output to the signal line 113 from the signal processing unit 103 of the pixel selected by the vertical scanning circuit unit 110 for the selected column.

[0037] The signal output to the signal line 113 is output via an output circuit 114 to a recording unit or a signal processing unit outside the photoelectric conversion device 100 .

[0038] 2, the pixels 101 may be arranged one-dimensionally in the pixel region 12. The function of the signal processing unit 103 does not necessarily need to be provided for each pixel 101, and for example, one signal processing unit 103 may be shared by a plurality of pixels 101 to perform signal processing sequentially.

[0039] Figures 4(A) and 4(B) are examples of block diagrams including the equivalent circuits of Figures 2 and 3. Figure 4(A) is an example in which the signal generating unit 215 is provided in common to a plurality of pixels, and Figure 4(B) is an example in which the control signal P_CLK can be controlled for each pixel. Figure 4(A) will be explained.

[0040] In FIG. 4A, a photoelectric conversion element 102 having an APD 201 is provided on a sensor substrate 11, and other members are provided on a circuit substrate 21.

[0041] The APD 201 generates charge pairs according to the incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. A reverse bias voltage that causes the APD 201 to perform avalanche multiplication is supplied to the anode and cathode. With such a voltage supplied, the charges generated by the incident light undergo avalanche multiplication, generating an avalanche current.

[0042] When a reverse bias voltage is supplied, there are two modes: Geiger mode, which operates with the potential difference between the anode and cathode greater than the breakdown voltage, and linear mode, which operates with the potential difference between the anode and cathode close to or less than the breakdown voltage. An APD that operates in Geiger mode is called a SPAD. For example, the voltage VL (first voltage) is -30V, and the voltage VH (second voltage) is 1V. The APD201 operates in linear mode. In the case of a SPAD, the potential difference is larger than in a linear mode APD, and the effect of the withstand voltage becomes more pronounced.

[0043] The switch 202 is connected to a power supply line to which a drive voltage VH is supplied and to the APD 201. The switch 202 is connected to one of the anode and cathode nodes of the APD 201. The switch 202 switches the potential difference between the anode and cathode of the APD 201 between a first potential difference that causes avalanche multiplication and a second potential difference that does not cause avalanche multiplication. Hereinafter, switching from the second potential difference to the first potential difference is also referred to as turning on the switch 202, and switching from the first potential difference to the second potential difference is also referred to as turning off the switch 202. The switch 202 functions as a quenching element. The switch 202 functions as a load circuit (quenching circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 201 (quenching operation). The switch 202 also has a function of returning the voltage supplied to the APD 201 to the drive voltage VH by passing a current equivalent to the voltage drop caused by the quench operation (recharge operation). In other words, the switch 202 functions as a control circuit for controlling the occurrence of avalanche multiplication in the APD 201.

[0044] The switch 202 can be configured, for example, by a MOS transistor. A control signal P_CLK for the switch 202, which is supplied from the signal generating unit 215, is applied to a gate electrode of the MOS transistor that configures the switch 202. In this embodiment, the on / off of the switch 202 is controlled by controlling the voltage applied to the gate electrode of the switch 202.

[0045] The signal processing section 103 has a waveform shaping section 210, a counter circuit 211, and a selection circuit 212. In this specification, it is sufficient that the signal processing section 103 has at least one of the waveform shaping section 210, the counter circuit 211, and the selection circuit 212.

[0046] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. The node on the input side of the waveform shaping unit 210 is nodeA, and the node on the output side is nodeB. The waveform shaping unit 210 changes the output potential from the node nodeB depending on whether the input potential to the node nodeA is equal to or higher than a predetermined value or lower. For example, in FIG. 5, when the input potential to the node nodeA becomes a high potential equal to or higher than the judgment threshold, the output potential from the node nodeB becomes a low level. When the input potential to the node nodeA becomes a potential lower than the judgment threshold, the output potential from the node nodeB becomes a high level. For example, an inverter circuit is used as the waveform shaping unit 210. In FIG. 4(A), an example in which one inverter is used as the waveform shaping unit 210 is shown, but a circuit in which multiple inverters are connected in series may be used, or other circuits having a waveform shaping effect may be used.

[0047] It is possible to perform a quench operation and a recharge operation using the switch 202 according to the avalanche multiplication in the APD 201, but depending on the timing of photon detection, the charge generated in the APD 201 may not be determined as an output signal. For example, assume that avalanche multiplication occurs in the APD 201, the node nodeA becomes low level, and a recharge operation is performed. In general, the decision threshold of the waveform shaping unit 210 is set to a potential higher than the potential difference at which avalanche multiplication occurs in the APD 201. When a photon is incident on the APD 201 when the potential of the node nodeA is lower than the decision threshold due to the recharge operation and is at a potential at which avalanche multiplication is possible in the APD 201, avalanche multiplication occurs in the APD 201 and the voltage of the nodeA drops. In other words, since the potential of the nodeA drops at a voltage lower than the decision threshold, no potential change occurs across the decision threshold, and the output potential from the node nodeB does not change. Therefore, even though avalanche multiplication occurs, photon detection is not detected as a signal. In particular, under high illuminance, photons enter the APD 201 continuously in a short period of time, making it difficult to determine the incident light as a signal. This can easily cause a discrepancy between the actual number of incident photons and the output signal, even under high illuminance.

[0048] In contrast, by applying a control signal P_CLK to the switch 202 to switch the switch 202 between an on state and an off state, it is possible to determine a signal even when photons continuously enter the APD 201 in a short period of time. In FIG. 5, an example is described in which the control signal P_CLK is a pulse signal with a repetitive cycle. In other words, in FIG. 5, a form is described in which the switch 202 is switched on and off at a predetermined clock frequency. However, the effect of suppressing an increase in power consumption of the photoelectric conversion device 100 can be obtained even if the pulse signal is not a signal with a repetitive cycle.

[0049] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 and holds the count value. When a control pulse pRES is supplied via a drive line 213, the signal held in the counter circuit 211 is reset.

[0050] A control pulse pSEL is supplied to the selection circuit 212 from the vertical scanning circuit unit 110 in Fig. 3 via the drive line 214 in Fig. 4(A), and switches between electrical connection and non-connection between the counter circuit 211 and the signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal.

[0051] The electrical connection may be switched by disposing a switch such as a transistor between the switch 202 and the APD 201 or between the photoelectric conversion element 102 and the signal processing unit 103. Similarly, the supply of the voltage VH or the voltage VL to the photoelectric conversion element 102 may be electrically switched using a switch such as a transistor.

[0052] In this embodiment, a configuration using the counter circuit 211 has been shown. However, instead of the counter circuit 211, the photoelectric conversion device 100 may acquire the pulse detection timing using a time-to-digital converter (hereinafter, referred to as TDC) or a memory. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. In order to measure the timing of the pulse signal, a control pulse pREF (reference signal) is supplied to the TDC from the vertical scanning circuit unit 110 in FIG. 3 via a drive line. The TDC acquires, as a digital signal, a value obtained by converting the input timing of the signal output from each pixel into a relative time based on the control pulse pREF.

[0053] FIG. 5 is a diagram showing a schematic diagram of the relationship between the control signal P_CLK of the switch, the potential of the node nodeA, the potential of the node nodeB, and the output signal. In this embodiment, when the control signal P_CLK is at a high level, the drive voltage VH is not easily supplied to the APD 201, and when the control signal P_CLK is at a low level, the drive voltage VH is supplied to the APD 201. The high level of the control signal P_CLK is, for example, 1V, and the low level of the control signal P_CLK is, for example, 0V. When the control signal P_CLK is at a high level, the switch is turned off, and when the control signal P_CLK is at a low level, the switch is turned on. The resistance value of the switch when the control signal P_CLK is at a high level is higher than the resistance value of the switch when the control signal P_CLK is at a low level. When the control signal P_CLK is at a high level, even if avalanche multiplication occurs in the APD 201, a recharge operation is not easily performed, so the potential supplied to the APD 201 is a potential equal to or lower than the breakdown voltage of the APD 201. Therefore, the avalanche multiplication operation in the APD 201 is stopped.

[0054] As shown in FIG. 4A, the switch 202 is configured with one transistor. It is preferable to use a single transistor to perform the quenching and recharging operations. This allows the number of circuits to be reduced compared to when the quenching and recharging operations are performed by different circuit elements. In particular, when each pixel has a counter circuit and the SPAD signal is read out for each pixel, it is preferable to reduce the circuit area used for the switch in order to place the counter circuit, and the effect of configuring the switch 202 with a single transistor is remarkable.

[0055] At time t1, the control signal P_CLK changes from high level to low level, the switch is turned on, and the recharge operation of the APD 201 is started. This causes the potential of the cathode of the APD 201 to transition to high level. Then, the potential difference between the potentials applied to the anode and cathode of the APD 201 becomes a state in which avalanche multiplication is possible. The potential of the cathode is the same as that of the node nodeA. Therefore, when the potential of the cathode transitions from low level to high level, the potential of the node nodeA becomes equal to or higher than the judgment threshold at time t2. At this time, the pulse signal output from the node nodeB is inverted and goes from high level to low level. After that, the potential difference of the drive voltage VH-drive voltage VL is applied to the APD 201. The control signal P_CLK becomes high level, and the switch is turned off.

[0056] Next, at time t3, when a photon is incident on the APD 201, avalanche multiplication occurs in the APD 201, and the voltage of the cathode drops. That is, the voltage of the node nodeA drops. When the amount of voltage drop becomes larger and the voltage difference applied to the APD 201 becomes smaller, the avalanche multiplication of the APD 201 stops as at time t2, and the voltage level of the node nodeA does not drop by more than a certain value. When the voltage of the node nodeA becomes lower than the decision threshold while the voltage of the node nodeA is dropping, the voltage of the node nodeB goes from low level to high level. That is, the part of the output waveform at the node nodeA that exceeds the decision threshold is waveform-shaped by the waveform shaping unit 210 and output as a signal at the nodeB. Then, it is counted by the counter circuit, and the count value of the counter signal output from the counter circuit increases by 1 LSB.

[0057] Between time t3 and time t4, photons are incident on the APD 201. However, since the switch is in the off state and the voltage applied to the APD 201 does not have a potential difference that allows avalanche multiplication, the voltage level of the node A does not exceed the decision threshold.

[0058] At time t4, the control signal P_CLK changes from high to low, and the switch is turned on. As a result, a current flows to node A to compensate for the voltage drop from the drive voltage VH, and the voltage of node A transitions to its original voltage level. At this time, the voltage of node A becomes equal to or higher than the decision threshold at time t5, so the pulse signal of node B is inverted and goes from high to low.

[0059] At time t6, node A is stabilized to the original voltage level, and the control signal P_CLK goes from low to high. Therefore, the switch is turned off. After this, the potentials of the nodes and signal lines change in response to the control signal P_CLK and the incidence of photons, as described from time t1 to time t6.

[0060] In the following, distance measuring devices according to the respective embodiments to which the above-described photoelectric conversion device is applied will be described.

[0061] (First embodiment) The distance measuring device will be described with reference to Figs. 6 to 9. Figs. 6 to 9 exemplify a plan view and a cross-sectional view of one of the multiple pixels 101 included in the pixel region 12. The pixel 101 has two sub-pixels 101a and 101b, which share one microlens 323 arranged on the pixel. Each of the sub-pixels 101a and 101b includes a photodiode. In the description, the photodiodes in the subpixels 101a and 101b are APDs in which avalanche multiplication of electric charges occurs. The distance measuring device can detect weak light at the single photon level by utilizing the avalanche multiplication of the APD in the pixel 101.

[0062] Fig. 6 is a plan view of a pixel 101 of the distance measuring device. Fig. 6 illustrates structures for explaining positional relationships in a planar view, and includes structures that do not exist on the same plane. Also, structures that are not used in the explanation are omitted from the illustration.

[0063] In the example of FIG. 6, pixel 101 has two subpixels 101a and 101b, and subpixels 101a and 101b share one microlens 323. Note that the subpixels included in pixel 101 are not limited to two subpixels arranged in one row and two columns. Pixel 101 may include four subpixels arranged in two rows and two columns, or may include more than four subpixels. The multiple subpixels included in pixel 101 share one microlens 323. When pixel 101 includes four or more subpixels, phase difference detection is performed in multiple directions, enabling more accurate image plane phase difference detection.

[0064] 6, the APD included in each of the subpixels 101a and 101b has a first semiconductor region 311 of a first conductivity type and a second semiconductor region 312 of a second conductivity type (not shown). If the APD of the subpixel 101a is a first avalanche photodiode and the APD of the subpixel 101b is a second avalanche photodiode, the first semiconductor region 311 of the first avalanche photodiode has a major axis direction and a minor axis direction. In other words, the first semiconductor region of the first avalanche photodiode has a length in a first direction and a length in a second direction perpendicular to the first direction, and the length in the first direction is longer than the length in the second direction.

[0065] Each of the APDs included in the pixel 101 has one or more cathode electrodes 301 and one or more anode electrodes 302. The cathode electrode 301 supplies a first voltage (cathode voltage) to the first semiconductor region 311 via a first contact. A first isolation portion 324a is arranged between the pixels 101, and a second isolation portion 324b is arranged between the APDs. A third semiconductor region 313 of a second conductivity type is provided in a region adjacent to the isolation portion 324 including the first isolation portion 324a and the second isolation portion 324b. The third semiconductor region 313 and the anode electrode 302 are electrically connected via a second contact, and a second voltage (anode voltage) is supplied from the anode electrode 302 to the third semiconductor region 313.

[0066] 7 to 9, details of each semiconductor region disposed in the semiconductor layer 300 will be described. Figures 7 to 9 respectively show an AA' cross-sectional view, a BB' cross-sectional view, and a CC' cross-sectional view of the pixel 101 shown in Figure 6.

[0067] Fig. 7 is an A-A' cross-sectional view of the pixel 101 shown in Fig. 6. Fig. 7 shows, in addition to the semiconductor layer 300, a pinning film 321, a planarizing layer 322, and a microlens 323 formed on the rear surface side of the substrate, and a cathode electrode 301 and a part of its wiring formed on the front surface side of the substrate and connected to the semiconductor layer 300.

[0068] 7, a seventh semiconductor region 317 of a first conductivity type is disposed around the first semiconductor region 311, and a second semiconductor region 312 of a second conductivity type is disposed adjacent to the first semiconductor region 311 in the depth direction. Furthermore, a fifth semiconductor region 315 is disposed on the back surface side of the second semiconductor region 312.

[0069] The first semiconductor region 311 and the second semiconductor region 312 form a PN junction. When a predetermined reverse bias voltage is applied to the first semiconductor region 311 and the second semiconductor region 312, the APD causes avalanche multiplication. A fifth semiconductor region 315 (for example, an epitaxial layer of the first conductivity type or an epitaxial layer of the second conductivity type) having a lower impurity concentration of the second conductivity type compared to the second semiconductor region 312 is provided in a region on the back surface side of the semiconductor layer 300 than the second semiconductor region 312. Therefore, by applying a reverse bias to this PN junction, a depletion layer spreads toward the back surface side of the semiconductor layer 300.

[0070] The seventh semiconductor region 317 is disposed so that at least a portion thereof contacts an end of the first semiconductor region 311. When the end of the first semiconductor region 311 becomes a strong electric field, edge breakdown, in which breakdown occurs at a low voltage in that region, is suppressed.

[0071] Near the interface between the separation portion 324 and the semiconductor layer 300, there are many dangling bonds, which are unbonded bonds of silicon, and dark current may occur through defect levels. In order to suppress the generation of this dark current, a third semiconductor region 313 of the second conductivity type is arranged so as to be in contact with the separation portion 324. For the same reason, a fourth semiconductor region 314 of the second conductivity type is arranged on the back surface side of the semiconductor layer 300. In addition, by forming the pinning film 321 at the back surface side interface of the semiconductor layer 300, holes are induced on the semiconductor layer 300 side, and dark current is suppressed.

[0072] Fig. 8 is a cross-sectional view taken along line B-B' of pixel 101 shown in Fig. 6. As shown in Fig. 8, pixel 101 is configured such that two APDs share one microlens 323. By sharing one microlens 323 among multiple APDs, it becomes possible to capture a light beam that has passed through a certain area of ​​the objective lens with each APD and to detect the amount of defocus and its direction from the output difference between the APDs below microlens 323. This realizes image plane phase difference autofocus that enables both imaging and phase difference detection.

[0073] Fig. 9 is a CC' cross-sectional view of the pixel 101 shown in Fig. 6. A cathode electrode 301 and an anode electrode 302 are disposed on the front side of the semiconductor layer 300. The cathode electrode 301 is electrically connected to the first semiconductor region 311 via a first contact, and the anode electrode 302 is electrically connected to the third semiconductor region 313 via a second contact.

[0074] 6 and 9 show an example in which the anode electrodes 302 are arranged at the four corners of two APDs arranged in one row and two columns. The anode electrodes 302 may be arranged at the four corners of one APD, or may be arranged for more APDs. The anode electrodes 302 may be arranged at the four corners of every four pixels of a pixel having two APDs arranged in one row and two columns, for example. In addition, since a plurality of APDs share a common anode potential, a part of the separation portion 324 may be formed to be separated. In other words, the semiconductor layer 300 may have a shape in which a part of the semiconductor layer 300 is connected to an adjacent pixel in a plan view.

[0075] The functional configuration of the distance measuring device 1000 will be described with reference to Fig. 10. Fig. 10 shows an example of a functional block diagram of the distance measuring device 1000. The distance measuring device 1000 includes a light emitting unit 1020, a driving unit 1030, a control signal generating unit 1040, a light receiving unit 1050, a distance measuring unit 1060, and an external I / F unit 1070. The distance measuring unit 1060 includes a time-to-digital conversion circuit (TDC) unit 1061, a histogram generating unit 1062, and a distance acquiring unit 1063.

[0076] The light emitting unit 1020 is an LED (Light Emitting Diode) that irradiates light onto the object 1010 that is the object of distance measurement. The light receiving unit 1050 is a pixel region 12 (pixel array), and transmits signal information obtained by receiving reflected light of light emitted from the light emitting unit 1020 to the distance measuring unit 1060. The driving unit 1030 drives the light emitting unit 1020 based on a control signal supplied from the control signal generating unit 1040. The control signal generating unit 1040 generates a control signal for driving the light emitting unit 1020 in synchronization with the light receiving timing of the light receiving unit 1050, and transmits the control signal to the driving unit 1030. The light receiving unit 1050 is a pixel region 12 (pixel array), and transmits signal information obtained by receiving reflected light of light emitted from the light emitting unit 1020 to the distance measuring unit 1060.

[0077] The distance measuring unit 1060 acquires the distance to the object 1010 based on the difference between the outputs of the sub-pixels that receive the reflected light. Specifically, the distance measuring unit 1060 calculates the distance from the distance measuring device 1000 to the object 1010 using a time-digital conversion circuit unit 1061, a histogram generating unit 1062, and a distance acquiring unit 1063 based on the signal information of the reflected light received by the light receiving unit 1050. In this embodiment, the distance measuring unit 1060 is configured to perform distance measurement using a dToF method that measures the time difference between irradiating light and detecting reflected light.

[0078] The time-to-digital conversion circuit unit 1061 converts the signal information of the reflected light into time information from when the light-emitting unit 1020 emits light to when the light-receiving unit 1050 receives the reflected light. The histogram generation unit 1062 generates a histogram that represents the distribution of the intensity of the reflected light versus the time from when the light-emitting unit 1020 starts emitting light. The intensity of the reflected light is represented, for example, by the count number of a pulse signal detected from the reflected light. The histogram generation unit 1062 generates a histogram that represents the relationship between the time from when the light-emitting unit 1020 emits light and the count value (the count number of the pulse signal of the reflected light).

[0079] The distance acquisition unit 1063 acquires the distance from the distance measuring device 1000 to the object 1010 based on the histogram generated by the histogram generation unit 1062. The distance acquisition unit 1063 can acquire the distance to the object 1010 by two methods, the ToF method and the phase difference method, using the histogram representing the intensity of reflected light.

[0080] A method of calculating the distance to an object 1010 using two methods will be described with reference to Figs. 11(A), 11(B) and 12(A) to 12(C). Figs. 11(A) and 11(B) are diagrams illustrating histograms representing the intensity of reflected light. The histograms representing the intensity of reflected light are generated when measuring distance using the ToF method. Figs. 12(A) to 12(C) are diagrams explaining distance measurement using the phase difference method.

[0081] The ToF method is a method for acquiring the distance to the object 1010 based on the time taken for the light emitted by the light emitting unit 1020 to be reflected by the object 1010 and return. The distance acquiring unit 1063 can acquire the time taken for the light to be reflected by the object 1010 and return from the histogram generated by the histogram generating unit 1062.

[0082] 11(A) and 11(B), the horizontal axis indicates the time since the light emitting unit 1020 emitted light. The vertical axis indicates the count value of the pulse signal of the reflected light received by the light receiving unit 1050 at each time. The distance acquiring unit 1063 can acquire the distance to the object 1010 based on the time at which the intensity (count value) of the reflected light peaks in the histogram.

[0083] 11A, the time dt at which the count value reaches a peak Ca is the time from when the light emitting unit 1020 emits light to when the light reflected by the object 1010 is received by the light receiving unit 1050. In other words, the time dt is the round trip time from when the light is emitted to when it is reflected by the object 1010 and returns. If the speed of light is c, the distance D from the distance measuring device 1000 to the object 1010 can be obtained by the following formula 1. D=c×dt / 2 (Formula 1)

[0084] The phase difference method is a method in which each of a plurality of pixels (sub-pixels) included in an image sensor having a DAF function receives reflected light, and acquires the distance to an object based on the difference in output of each of the plurality of sub-pixels. The distance acquisition unit 1063 can acquire the distance to the object 1010 based on the phase difference of the reflected light received by each of the plurality of sub-pixels (the phase difference of the signal output from each pixel).

[0085] 12(A) to 12(C) are diagrams for explaining distance measurement using the phase difference method. 2(A) to 12(C) show a configuration in which light beams A and B passing through an imaging optical system 1201 are received by a pixel array in which multiple pixels 101, each including one microlens 323 and sub-pixels 101a and 101b that share the microlens 323, are arranged.

[0086] Fig. 12(A) shows a front-focus state where the focus is in front of an object 1010 as a subject. Fig. 12(B) shows a state where the focus is on the object 1010. Fig. 12(C) shows a back-focus state where the focus is behind the object 1010. It is possible to detect the amount of defocus and the direction of deviation in an image plane phase difference AF (autofocus) method based on the phase difference between the signal of the reflected light received by the sub-pixel 101a and the signal of the reflected light received by the sub-pixel 101b. The distance acquisition unit 1063 can measure the distance to the subject (object 1010) based on the amount of defocus and the direction of deviation in the image plane phase difference AF.

[0087] Further, the distance acquisition unit 1063 can detect the phase difference between the sub-pixels 101a and 101b using a histogram generated during distance measurement using the ToF method. For example, when the object 1010 is not in focus, the histograms of the reflected light received by the sub-pixels 101a and 101b are histograms with different intensities as shown in FIG. 11(A) and FIG. 11(B). This is because the sub-pixels 101a and 101b receive different amounts of light in the front focus state of FIG. 12(A) or the back focus state of FIG. 12(C). On the other hand, in the in-focus state of FIG. 12(B), the sub-pixels 101a and 101b receive light with approximately the same intensity. In this way, the distance acquisition unit 1063 can detect the phase difference between the sub-pixels 101a and 101b by acquiring the difference in the intensity of the reflected light between the two sub-pixels sharing the microlens 323 within a predetermined range of the pixel array.

[0088] The difference in reflected light intensity can be the difference in reflected light intensity (count value) at the peak of the histogram of each of the multiple sub-pixels. In this case, the distance acquisition unit 1063 can detect the phase difference between the sub-pixels 101a and 101b based on the difference between the reflected light intensity Ca at the peak of the histogram of the sub-pixel 101a and the reflected light intensity Cb at the peak of the histogram of the sub-pixel 101b.

[0089] The difference in reflected light intensity may be the difference in the sum of the reflected light intensity (count value) of the histograms of the sub-pixels. In this case, the distance acquisition unit 1063 can detect the phase difference between the sub-pixels 101a and 101b based on the difference between the sum of the reflected light intensity of the histogram for the sub-pixel 101a and the sum of the reflected light intensity of the histogram for the sub-pixel 101b. The sum of the reflected light intensity may be the sum of the count values ​​in a predetermined range of time including the peak.

[0090] 10 is an interface for communicating with an external device. The external I / F unit 1070 can output information on the distance to the object 1010 acquired by the distance measuring unit 1060 to the external device.

[0091] 13 is a flowchart illustrating the distance measurement process. In step S1301, the light emitting unit 1020 irradiates light onto the object 1010. In step S1302, the light receiving unit 1050 receives reflected light of the light irradiated in step S1301. In step S1303, the histogram generating unit 1062 generates a histogram indicating the intensity of the reflected light for each of the sub-pixels 101a and 101b that receive the reflected light in the light receiving unit 1050.

[0092] In step S1304, the distance acquisition unit 1063 acquires the distance to the object 1010 based on the return time of the reflected light. That is, the distance acquisition unit 1063 performs distance measurement using the ToF method. The distance acquisition unit 1063 may acquire the distance to the object 1010 using some of the sub-pixels that share the microlens 323. For example, the distance acquisition unit 10 63 can obtain the distance to the object 1010 using the histogram of either sub-pixel 101a or sub-pixel 101b.

[0093] Furthermore, the distance acquisition unit 1063 may acquire the distance to the object 1010 based on outputs from multiple sub-pixels among the sub-pixels sharing the microlens 323. For example, the distance acquisition unit 1063 can acquire the distance to the object 1010 based on the sum of the reflected light intensities (count values) between the histograms of two or more sub-pixels among the multiple sub-pixels.

[0094] In step S1305, the distance acquisition unit 1063 acquires the distance to the object 1010 based on the signal intensity difference between the subpixels 101a and 101b using the histogram generated in step S1303. That is, the distance acquisition unit 1063 performs distance measurement using a phase difference method.

[0095] In step S1306, the distance acquisition unit 1063 (corresponding to an output unit) outputs distance information regarding the distance to the object 1010 based on the distance to the object 1010 acquired in step S1304 and the distance to the object 1010 acquired in step S1305. The distance acquisition unit 1063 may output the distances acquired by the two methods respectively, or may output the distance acquired by the method with higher accuracy. Furthermore, the distance acquisition unit 1063 may output a distance obtained by averaging or weighted averaging the distances acquired by the two methods.

[0096] Each method of distance measurement has its own advantages and disadvantages, and one method of distance measurement may be more accurate than another method depending on a predetermined condition. The predetermined condition may be, for example, the distance to the object 1010, whether or not reflected light is detected, whether or not contrast of the object 1010 is detected, etc. Therefore, the distance acquisition unit 1063 may output the distance acquired by one of the methods depending on the predetermined condition as the distance to the object 1010. The distance information output depending on the predetermined condition will be described below.

[0097] When the distance to the object 1010 is greater than a predetermined threshold, the ToF method can measure distance more accurately than the phase difference method. Therefore, when either of the distances to the object 1010 acquired by the two methods is greater than a predetermined threshold, the distance acquisition unit 1063 outputs the distance acquired by the ToF method as distance information in step S1304. On the other hand, when both of the distances to the object 1010 acquired by the two methods are equal to or less than the predetermined threshold, the distance acquisition unit 1063 outputs the distance acquired by the phase difference method as distance information in step S1305. The predetermined threshold can be set in advance based on the actual measurement results of each method.

[0098] Furthermore, the method of comparing the distance to the object 1010 acquired by the two methods with the predetermined threshold is not limited to the above method. For example, the distance acquisition unit 1063 may compare the average of the distances to the object 1010 acquired by the two methods with the predetermined threshold, or may compare the distance to the object 1010 acquired by either of the two methods with the predetermined threshold.

[0099] Furthermore, if the reflected light from the object 1010 is detected, the distance acquisition unit 1063 outputs the distance acquired by the ToF method as distance information in step S1304. On the other hand, if the reflected light from the object 1010 is not detected, the distance acquisition unit 1063 outputs the distance acquired by the phase difference method as distance information in step S1305. If the reflected light from the object 1010 is not detected, it may be, for example, if the irradiated light is absorbed by the object 1010 or if the reflected light from the object 1010 is not received due to the influence of disturbance light.

[0100] In addition, in the case where the distance to the object 1010 can be obtained from the difference in the outputs of the multiple pixels that share the microlens 323, the distance acquisition unit 1063 performs The distance acquired by the phase difference method is output as distance information. On the other hand, if the distance to the object 1010 cannot be obtained from the difference in the outputs of the multiple pixels sharing the microlens 323, the distance acquisition unit 1063 outputs the distance acquired by the ToF method as distance information in step S1304. If a phase difference is not detected from the outputs of the multiple pixels, for example, the contrast between the object 1010 and the background is reduced.

[0101] According to the first embodiment, the distance measuring device 1000 can also acquire information on the phase difference from peak information of a histogram, etc., by configuring multiple sub-pixels 101a and 101b capable of distance measurement by the ToF method to share one microlens 323. In this way, the distance measuring device 1000 can realize distance measurement by multiple methods and prevent the distance measuring system from becoming large, with a simple configuration in which multiple sub-pixels share one microlens 323.

[0102] Distance measurement using the ToF method and distance measurement using the phase difference method differ in the objects that can be measured accurately and in the situations (distance to the object, ambient light, etc.) in which they can be measured accurately. Therefore, the distance measuring device 1000 can achieve more accurate distance measurement by combining distance measurements using these methods.

[0103] (Modification of the first embodiment) In a configuration in which multiple subpixels share one microlens 323, when measuring distances using the ToF method, the amount of light received by each subpixel may differ. Therefore, in a modified example, in step S1304 in Fig. 13, the distance acquisition unit 1063 acquires the distance to the object 1010 using the subpixel with the largest output among the multiple subpixels sharing the microlens 323.

[0104] 14(A) and 14(B), a description will be given of timing jitter that occurs in ToF distance measurement due to a configuration in which multiple subpixels share one microlens 323. As shown in Fig. 14(A), the amount of light incident on subpixel 101b is less than that on subpixel 101a. In subpixel 101b, photoelectric conversion occurs near the incident surface, so the travel distance of electrons to avalanche region 1400 where avalanche multiplication occurs becomes longer, and timing jitter (timing fluctuation on the time axis of the signal waveform) may worsen.

[0105] Therefore, the distance acquisition unit 1063 acquires the distance to the object 1010 based on the output of the subpixel 101a, which has a larger output due to a larger amount of incident light than the subpixel 101b. The subpixel with the largest output can be, for example, the subpixel with the largest count value at the peak of the histogram or the largest sum of the count values ​​of the histogram.

[0106] 14B shows a histogram 1401a of the subpixel 101a, a histogram 1401b of the subpixel 101b, and a histogram 1401 (the sum of the histograms 1401a and 1401b) of the pixel 101. In the subpixel 101b, the arrival time of electrons to the avalanche region 1400 is slower than in the subpixel 101a. The distance acquisition unit 1063 acquires the distance to the object 1010 using the histogram 1401a of the subpixel 101a without considering the histogram 1401b of the subpixel 101b, thereby suppressing the influence of timing jitter.

[0107] In a modified example of the first embodiment, the distance measuring device 1000 can suppress the effects of timing jitter by obtaining the distance to the object 1010 using the subpixel with the highest output among multiple subpixels that share the microlens 323.

[0108] Note that the distance measurement is not limited to being performed based on the output of any one of the multiple sub-pixels, and the distance measuring device 1000 may obtain the distance to the object 1010 based on a histogram generated by weighting the outputs of the multiple sub-pixels. The distance measuring device 1000 can perform weighting based on the ratio of the sum of the count values ​​of the sub-pixels, for example.

[0109] Second embodiment The distance measuring system according to this embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing a schematic configuration of the distance measuring system according to this embodiment.

[0110] The photoelectric conversion device (distance measuring device) described in the first embodiment is applicable to various distance measuring systems, and can measure the distance to an object to be measured, such as a subject. The distance measuring system includes at least the distance measuring device according to the above embodiment, and a signal processing unit that processes a signal output from the distance measuring device. Examples of devices to which such a distance measuring system can be applied include digital still cameras, digital camcorders, surveillance cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, sensors, and measuring instruments. Camera modules equipped with an optical system such as a lens and an imaging device are also included in devices to which the distance measuring system is applied. FIG. 15 illustrates a block diagram of a digital still camera as an example of these.

[0111] 15 includes an imaging device 1504 to which a distance measuring device, which is an example of a photoelectric conversion device, is applied, and a lens 1502 that forms an optical image of a subject on the imaging device 1504. The distance measuring system also includes an aperture 1503 for varying the amount of light passing through the lens 1502, and a barrier 1501 for protecting the lens 1502. The lens 1502 and the aperture 1503 are an optical system that focuses light on the imaging device 1504. The imaging device 1504 is the photoelectric conversion device (distance measuring device) of the first embodiment described above, and converts the optical image formed by the lens 1502 into an electrical signal.

[0112] The ranging system also has a signal processing unit 1507 which is an image generating unit that generates an image by processing an output signal output from the imaging device 1504. The signal processing unit 1507 performs various corrections and compression as necessary to output image data. The signal processing unit 1507 may be formed on a semiconductor substrate on which the imaging device 1504 is provided, or may be formed on a semiconductor substrate separate from the imaging device 1504. Moreover, the imaging device 1504 and the signal processing unit 1507 may be formed on the same semiconductor substrate.

[0113] The ranging system further includes a memory unit 1510 for temporarily storing image data, and an external interface unit (external I / F unit) 1513 for communicating with an external computer or the like. The ranging system further includes a recording medium 1512 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1511 for recording or reading out data on the recording medium 1512. The recording medium 1512 may be built into the ranging system, or may be removable.

[0114] The distance measuring system further includes an overall control / calculation unit 1509 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1508 that outputs various timing signals to the image pickup device 1504 and the signal processing unit 1507. Here, the timing signals and the like may be input from outside, and the distance measuring system only needs to include at least the image pickup device 1504 and the signal processing unit 1507 that processes the output signal output from the image pickup device 1504.

[0115] The imaging device 1504 outputs an imaging signal to the signal processing unit 1507. The signal processing unit 1507 performs predetermined signal processing on the imaging signal output from the imaging device 1504, and outputs image data. The signal processing unit 1507 generates an image using the imaging signal.

[0116] In this way, according to this embodiment, it is possible to realize a distance measuring system to which the photoelectric conversion device (distance measuring device) of the first embodiment is applied.

[0117] Third embodiment The distance measuring system and the moving object of this embodiment will be described with reference to Fig. 16(A) and Fig. 16(B). Fig. 16(A) is a diagram showing the configuration of the distance measuring system of this embodiment, and Fig. 16(B) is a diagram showing the configuration of the moving object of this embodiment.

[0118] FIG. 16A shows an example of a distance measurement system related to an in-vehicle camera. The distance measurement system 1600 has an imaging device 1610 to which the photoelectric conversion device (distance measurement device) described in the first embodiment is applied. The distance measurement system 1600 has an image processing unit 1612 that performs image processing on a plurality of image data acquired by the imaging device 1610. The distance measurement system 1600 also has a distance acquisition unit 1616 that calculates the distance to an object, and a collision determination unit 1618 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the distance acquisition unit 1616 may acquire distance information to the object by ToF (Time Of Flight), or may acquire distance information using parallax information or the like. The distance acquisition unit 1616 may also acquire distance information by combining distance measurement by ToF and distance measurement based on the phase difference between pixels. That is, the distance information is information related to parallax, defocus amount, distance to the object, and the like. The collision determination unit 1618 may use any of these distance information to determine the possibility of collision. The distance information acquisition means may be realized by dedicated hardware, a software module, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a combination of these.

[0119] The distance measuring system 1600 is connected to a vehicle information acquisition device 1620, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The distance measuring system 1600 is also connected to an ECU 1630, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 1618. The distance measuring system 1600 is also connected to an alarm device 1640 that issues an alarm to the driver based on the judgment result of the collision judgment unit 1618. For example, if the judgment result of the collision judgment unit 1618 indicates that there is a high possibility of a collision, the ECU 1630 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 1640 warns the user by sounding an alarm, displaying alarm information on the screen of a car navigation system, etc., or vibrating a seat belt or steering wheel.

[0120] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the distance measuring system 1600. Fig. 16(B) shows a distance measuring system for imaging the area in front of the vehicle (imaging range 1650). A vehicle information acquisition device 1620 sends an instruction to the distance measuring system 1600 or the imaging device 1610. This configuration can further improve the accuracy of distance measurement.

[0121] Although an example of control to avoid collision with other vehicles has been described above, the present invention can also be applied to control of automatic driving to follow other vehicles, and control of automatic driving to avoid going out of lanes. Furthermore, the distance measurement system is not limited to vehicles such as automobiles, but can be applied to moving bodies (mobile devices) such as ships, aircraft, and industrial robots. The moving body includes one or both of a driving force generating unit that generates a driving force mainly used to move the moving body, and a rotating body mainly used to move the moving body. The driving force generating unit can be an engine, a motor, etc. The rotating body can be a tire, a wheel, a screw of a ship, a propeller of an aircraft, etc. In addition, the present invention can be applied to a wide range of systems that recognize objects, not limited to moving bodies, such as Intelligent Transport Systems (ITS). It can be applied to the equipment used.

[0122] (Fourth embodiment) The distance measuring system of this embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the configuration of a range image sensor which is the distance measuring system of this embodiment.

[0123] 17, the distance image sensor 1701 is configured to include an optical system 1707, a photoelectric conversion device 1708, an image processing circuit 1704, a monitor 1705, and a memory 1706. The distance image sensor 1701 can obtain a distance image according to the distance to the subject by receiving light (modulated light or pulsed light) that is projected from a light source device 1709 toward the subject and reflected from the surface of the subject.

[0124] The optical system 1707 is configured to have one or more lenses, and guides image light (incident light) from a subject to a photoelectric conversion device 1708 , forming an image on the light receiving surface (sensor portion) of the photoelectric conversion device 1708 .

[0125] The photoelectric conversion device 1708 is the photoelectric conversion device (distance measuring device) of the first embodiment described above, and a distance signal indicating a distance determined from a received light signal output from the photoelectric conversion device 1708 is supplied to the image processing circuit 1704.

[0126] The image processing circuit 1704 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 1708. The distance image (image data) obtained by this image processing is then supplied to a monitor 1705 for display, or supplied to a memory 1706 for storage (recording).

[0127] In the range image sensor 1701 configured in this manner, by applying the photoelectric conversion device (range measuring device) of the first embodiment described above, it is possible to obtain, for example, a more accurate range image as the range measuring accuracy improves.

[0128] Fifth embodiment The distance measuring system of this embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of a schematic configuration of an endoscopic surgery system which is the distance measuring system of this embodiment.

[0129] 18 shows a state in which an operator (doctor) 1831 is performing surgery on a patient 1832 on a patient bed 1833 using an endoscopic surgery system 1850. As shown in the figure, the endoscopic surgery system 1850 is composed of an endoscope 1800, a surgical tool 1810, and a cart 1834 on which various devices for endoscopic surgery are mounted.

[0130] The endoscope 1800 is composed of a lens barrel 1801, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1832, and a camera head 1802 connected to the base end of the lens barrel 1801. In the illustrated example, the endoscope 1800 is configured as a so-called rigid lens barrel having a rigid lens barrel 1801, but the endoscope 1800 may be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0131] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1801. A light source device 1803 is connected to the endoscope 1800, and light generated by the light source device 1803 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 1801, and is irradiated via the objective lens toward an observation target in a body cavity of a patient 1832. The endoscope 1800 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0132] An optical system and a photoelectric conversion device are provided inside the camera head 1802, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device may be the photoelectric conversion device (range measuring device) described in the first embodiment. The image signal is transmitted to a camera control unit (CCU) 1835 as RAW data.

[0133] The CCU 1835 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1800 and the display device 1836. Furthermore, the CCU 1835 receives an image signal from the camera head 1802, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0134] Under the control of the CCU 1835 , the display device 1836 displays an image based on an image signal that has been subjected to image processing by the CCU 1835 .

[0135] The light source device 1803 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1800 with irradiation light when photographing an operation site or the like.

[0136] The input device 1837 is an input interface for the endoscopic surgery system 1850. A user can input various information and instructions to the endoscopic surgery system 1850 via the input device 1837.

[0137] The treatment tool control device 1838 controls the driving of the energy treatment tool 1812 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0138] The light source device 1803 that supplies irradiation light to the endoscope 1800 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1803. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 1802 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0139] Furthermore, the light source device 1803 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1802 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0140] Furthermore, the light source device 1803 may be configured to be capable of supplying light in a predetermined wavelength band corresponding to special light observation. Special light observation utilizes, for example, the wavelength dependency of light absorption in body tissue. Specifically, by irradiating light of a narrower band than the light (i.e., white light) irradiated during normal observation, a predetermined tissue such as blood vessels on the surface of the mucosa is photographed with high contrast. Alternatively, special light observation may involve fluorescence observation in which an image is obtained from fluorescence generated by irradiating excitation light. In fluorescence observation, excitation light is irradiated onto body tissue, and fluorescence from the body tissue is detected. It is possible to observe light by using a special light, or to locally inject a reagent such as indocyanine green (ICG) into a body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1803 can be configured to supply narrowband light and / or excitation light corresponding to such special light observation.

[0141] Sixth embodiment The distance measuring system of this embodiment will be described with reference to Figs. 19(A) and 19(B). Fig. 19(A) describes glasses 1900 (smart glasses) which are the distance measuring system of this embodiment. The glasses 1900 have a photoelectric conversion device 1902. The photoelectric conversion device 1902 is the photoelectric conversion device (distance measuring device) described in the first embodiment. A display device including a light emitting device such as an OLED or LED may be provided on the rear side of the lens 1901. The photoelectric conversion device 1902 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1902 is not limited to that shown in Fig. 19(A).

[0142] The glasses 1900 further include a control device 1903. The control device 1903 functions as a power source that supplies power to the photoelectric conversion device 1902 and the display device. The control device 1903 also controls the operations of the photoelectric conversion device 1902 and the display device. The lens 1901 is formed with an optical system for focusing light on the photoelectric conversion device 1902.

[0143] FIG. 19(B) illustrates glasses 1910 (smart glasses) according to one application example. The glasses 1910 have a control device 1912, and the control device 1912 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1902 and a display device. The lens 1911 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 1912 and the display device, and an image is projected onto the lens 1911. The control device 1912 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used for detecting the line of sight. The infrared light emitting unit emits infrared light toward the eyeball of a user gazing at a display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By having a reduction means for reducing light from the infrared light emitting unit to the display unit in a planar view, deterioration of image quality is reduced.

[0144] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0145] More specifically, the gaze detection process is performed based on the pupil-corneal reflex method. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0146] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information about the user's line of sight from the photoelectric conversion device.

[0147] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0148] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first display area and the second display area may be determined by a control device of the display device, or may be determined by an external control device and received. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0149] AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. When included in the external device, the estimation result by the AI ​​is transmitted to the display device via communication.

[0150] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0151] Seventh embodiment The photoelectric conversion device (range measuring device) and the range measuring system according to the first embodiment may be applied to electronic devices such as so-called smartphones and tablets.

[0152] 20(A) and 20(B) are diagrams showing an example of an electronic device 2000 equipped with a photoelectric conversion device. Fig. 20(A) shows the front side of the electronic device 2000, and Fig. 20(B) shows the rear side of the electronic device 2000.

[0153] 20(A), a display 2010 for displaying an image is disposed in the center of the surface of the electronic device 2000. Then, along the upper side of the surface of the electronic device 2000, front cameras 2021 and 2022 using photoelectric conversion devices, an IR light source 2030 for emitting infrared light, and a visible light source 2040 for emitting visible light are disposed.

[0154] Also, as shown in FIG. 20(B), along the upper edge of the back surface of the electronic device 2000, rear cameras 2051 and 2052 using photoelectric conversion devices, an IR light source 2060 that emits infrared light, and a visible light source 2070 that emits visible light are arranged.

[0155] In the electronic device 2000 configured in this manner, by applying the above-mentioned photoelectric conversion device, for example, it is possible to capture a higher quality image and measure the distance to a subject with high accuracy. The photoelectric conversion device can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using active infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy and performance of these electronic devices.

[0156] Eighth embodiment FIG. 21 is a block diagram of an X-ray CT apparatus in this embodiment. The photoelectric conversion device (distance measuring device) and distance measuring system of the above-mentioned first embodiment are applicable to the detector of the X-ray CT apparatus. The X-ray CT apparatus 2100 in this embodiment includes an X-ray generation unit 2110, a wedge 2111, a collimator 2112, an X-ray detection unit 2120, and a tabletop 2130. The X-ray CT apparatus 2100 further includes a rotating frame 2140, a high-voltage generator 2150, a data acquisition system (DAS) 2151, a signal processing unit 2152, a table 2130, and a scanning electron microscope 2140. It has a display unit 2153 and a control unit 2154.

[0157] The X-ray generating unit 2110 is composed of, for example, a vacuum tube that generates X-rays. A high voltage and a filament current are supplied to the vacuum tube of the X-ray generating unit 2110 from a high voltage generator 2150. X-rays are generated by irradiating thermal electrons from the cathode (filament) toward the anode (target).

[0158] The wedge 2111 is a filter that adjusts the amount of X-rays irradiated from the X-ray generation unit 2110. The wedge 2111 attenuates the amount of X-rays so that the X-rays irradiated from the X-ray generation unit 2110 to the subject have a predetermined distribution. The collimator 2112 is made of a lead plate or the like that narrows down the irradiation range of the X-rays that have passed through the wedge 2111. The X-rays generated by the X-ray generation unit 2110 are shaped into a cone beam via the collimator 2112 and irradiated to the subject on the tabletop 2130.

[0159] The X-ray detection unit 2120 is configured using the above-mentioned photoelectric conversion device or distance measurement system. The X-ray detection unit 2120 detects X-rays emitted from the X-ray generation unit 2110 and transmitted through the subject, and outputs a signal corresponding to the X-ray dose to the DAS 2151.

[0160] The rotating frame 2140 has an annular shape and is configured to be rotatable. An X-ray generation unit 2110 (wedge 2111, collimator 2112) and an X-ray detection unit 2120 are arranged facing each other inside the rotating frame 2140. The X-ray generation unit 2110 and the X-ray detection unit 2120 are rotatable together with the rotating frame 2140.

[0161] The high-voltage generator 2150 includes a boost circuit, and outputs a high voltage to the X-ray generation unit 2110. The DAS 2151 includes an amplifier circuit and an A / D conversion circuit, and outputs a signal from the X-ray detection unit 2120 to the signal processing unit 2152 as digital data.

[0162] The signal processing unit 2152 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and is capable of executing image processing on digital data. The display unit 2153 includes a flat display device, and is capable of displaying X-ray images. The control unit 2154 includes a CPU, a ROM, a RAM, and the like, and controls the operation of the entire X-ray CT device 2100.

[0163] Ninth embodiment A distance measuring system according to the ninth embodiment will be described with reference to Fig. 22. Fig. 22 is a block diagram showing a schematic configuration of an imaging system SYS which is a distance measuring system according to the ninth embodiment. The imaging system SYS includes at least a distance measuring device according to the above embodiment and a signal processing unit which processes a signal output from the distance measuring device.

[0164] The imaging system SYS is an information terminal having a camera and a photographing function. The imaging system SYS is constructed using an imaging device IS. The imaging device IS may further include a package PKG that houses an imaging device IC. The package PKG may include a base to which the imaging device IC is fixed, and a lid that faces the imaging device IC. The package PKG may include a connection member (a member that connects a terminal provided on the base with a terminal provided on the imaging device IC). The imaging device IS may also mount a plurality of imaging device ICs side by side in a common package PKG. Also, the imaging device IS may also mount an imaging device IC and another semiconductor device IC stacked on top of each other in a common package PKG.

[0165] The imaging system SYS may include an optical system OU (optical device) that forms an image on the imaging device IS. The imaging system SYS also includes a control device CU, a processing device PU, a display device DU, a storage device M, and a The imaging device may include at least one of the control unit CU and the processing unit PU. The control unit CU controls the imaging device IS, the processing unit PU processes signals obtained from the imaging device IS, the display unit DU displays images obtained from the imaging device IS, and the storage unit MU stores images obtained from the imaging device IS.

[0166] (others) Although various devices have been described in the above embodiment, the camera may further include a mechanical device. The mechanical device in the camera can drive optical components for zooming, focusing, and shutter operation. Alternatively, the mechanical device in the camera can move a photoelectric conversion device for vibration reduction operation.

[0167] The device may be transportation equipment such as a vehicle, ship, or aircraft. A mechanical device in transportation equipment may be used as a moving device. The device as transportation equipment is suitable for transporting a photoelectric conversion device or for assisting and / or automating driving (piloting) by using an imaging function. A processing device for assisting and / or automating driving (piloting) can perform processing for operating the mechanical device as a moving device based on information obtained by the photoelectric conversion device.

[0168] The above-described embodiments can be modified as appropriate without departing from the technical concept. The disclosure of this specification includes not only what is described in this specification, but also all matters that can be understood from this specification and the drawings attached hereto. The disclosure of this specification also includes the complement of the concepts described in this specification. In other words, if this specification contains a statement that "A is greater than B," for example, this specification can be said to disclose that "A is not greater than B" even if the statement that "A is not greater than B" is omitted. This is because when it is stated that "A is greater than B," it is assumed that the case in which "A is not greater than B" is taken into consideration.

[0169] In the present specification, expressions such as "A or B", "at least one of A and B", "at least one of A or / and B", "one or more of A or / and B" may be used. In this case, unless otherwise expressly defined, all possible combinations of the listed items can be included. That is, the above expressions are understood to disclose all cases including at least one A, at least one B, and both at least one A and at least one B. This applies equally to combinations of three or more elements.

[0170] The disclosure of this embodiment includes the following configuration. (Configuration 1) A light emitting unit that irradiates light onto an object; A plurality of pixels; a microlens disposed on the plurality of pixels and shared by the plurality of pixels; a first distance measuring unit that obtains a distance to the object based on a time taken for the light emitted by the light emitting unit to be reflected and then received; a second distance measuring unit that obtains a distance to the object based on a difference between outputs of the plurality of pixels that receive the reflected light; A distance measuring device comprising: (Configuration 2) The first distance measuring unit obtains the distance to the object by using some of the plurality of pixels. 2. The distance measuring device according to configuration 1, (Configuration 3) The first distance measuring unit obtains the distance to the object using a pixel having the largest output among the plurality of pixels. 3. The distance measuring device according to configuration 2. (Configuration 4) The light emitting unit further includes a generation unit that generates a histogram that represents a distribution of the intensity of the reflected light with respect to time after the light emitting unit starts emitting light. 4. A distance measuring device according to any one of configurations 1 to 3. (Configuration 5) The intensity of the reflected light is represented by the count number of the pulse signal detected from the reflected light. 5. The distance measuring device according to configuration 4. (Configuration 6) The first distance measuring unit obtains the distance to the object based on the time when the intensity of the reflected light reaches a peak in the histogram. 6. A distance measuring device according to configuration 4 or 5. (Configuration 7) The first distance measuring unit obtains the distance to the object based on a sum of the intensity of the reflected light between the histograms of two or more pixels among the plurality of pixels. 6. A distance measuring device according to configuration 4 or 5. (Configuration 8) The second distance measuring unit acquires the distance to the object based on a difference between an intensity of the reflected light at a peak of the histogram for a first pixel among the plurality of pixels and an intensity of the reflected light at a peak of the histogram for a second pixel among the plurality of pixels. 8. A distance measuring device according to any one of configurations 4 to 7. (Configuration 9) The second distance measuring unit acquires the distance to the object based on a difference between a sum of the reflected light intensities of the histogram for a first pixel among the plurality of pixels and a sum of the reflected light intensities of the histogram for a second pixel among the plurality of pixels. 8. A distance measuring device according to any one of configurations 4 to 7. (Configuration 10) an output unit that outputs distance information regarding a distance to the object based on a first distance, which is the distance to the object acquired by the first distance measuring unit, and a second distance, which is the distance to the object acquired by the second distance measuring unit; 10. The distance measuring device according to any one of configurations 1 to 9, further comprising: (Configuration 11) The output unit is If the first distance or the second distance is longer than a predetermined threshold, output the first distance as the distance information; When the first distance and the second distance are equal to or smaller than a predetermined threshold, the second distance is output as the distance information. 11. A distance measuring device according to configuration 10. (Configuration 12) The output unit is When the reflected light from the object is detected, the first distance is output as the distance information; If the reflected light from the object is not detected, the second distance is output as the distance information. 11. A distance measuring device according to configuration 10. (Configuration 13) The output unit is When the distance to the object is obtained from the difference between the outputs of the plurality of pixels, outputting the second distance as the distance information; When the distance to the object cannot be obtained from the difference between the outputs of the plurality of pixels, the first distance is output as the distance information. 11. A distance measuring device according to configuration 10. (Configuration 14) Each of the pixels includes an avalanche photodiode. 14. A distance measuring device according to any one of configurations 1 to 13. (Configuration 15) A distance measuring device according to any one of configurations 1 to 14, a signal processing unit that processes a signal output by the distance measuring device; A ranging system comprising: (Configuration 16) A moving object including a distance measuring device according to any one of configurations 1 to 14, A control unit controls the movement of the moving body using information on the distance to the object acquired by the distance measuring device. A moving object characterized by: (Configuration 17) A distance measuring device according to any one of configurations 1 to 14, an optical device corresponding to the distance measuring device; A control device for controlling the distance measuring device; a processing device for processing a signal output from the distance measuring device; a display device for displaying information obtained by the distance measuring device; a storage device for storing information obtained by the distance measuring device; and a mechanical device that operates based on the information obtained by the distance measuring device; At least one of An apparatus comprising: [Explanation of symbols]

[0171] 101a, 101b: pixels (sub-pixels), 323: microlens 1020: Light emitting unit, 1060: Distance measuring unit

Claims

1. A light emitting unit that irradiates light onto an object; A plurality of pixels; a microlens disposed on the plurality of pixels and shared by the plurality of pixels; a first distance measuring unit that obtains a distance to the object based on a time taken for the light emitted by the light emitting unit to be reflected and then received; a second distance measuring unit that obtains a distance to the object based on a difference between outputs of the plurality of pixels that receive the reflected light; A distance measuring device comprising:

2. The first distance measuring unit obtains the distance to the object by using some of the plurality of pixels.

2. A distance measuring device according to claim 1.

3. The first distance measuring unit obtains the distance to the object using a pixel having the largest output among the plurality of pixels.

3. A distance measuring device according to claim 2.

4. The light emitting unit further includes a generation unit that generates a histogram that represents a distribution of the intensity of the reflected light with respect to time after the light emitting unit starts emitting light.

2. A distance measuring device according to claim 1.

5. The intensity of the reflected light is represented by the count number of the pulse signal detected from the reflected light.

5. A distance measuring device according to claim 4.

6. The first distance measuring unit obtains the distance to the object based on the time when the intensity of the reflected light reaches a peak in the histogram.

5. A distance measuring device according to claim 4.

7. The first distance measuring unit obtains the distance to the object based on a sum of the intensity of the reflected light between the histograms of two or more pixels among the plurality of pixels.

5. A distance measuring device according to claim 4.

8. The second distance measuring unit acquires the distance to the object based on a difference between an intensity of the reflected light at a peak of the histogram for a first pixel among the plurality of pixels and an intensity of the reflected light at a peak of the histogram for a second pixel among the plurality of pixels.

5. A distance measuring device according to claim 4.

9. The second distance measuring unit acquires the distance to the object based on a difference between a sum of the reflected light intensities of the histogram for a first pixel among the plurality of pixels and a sum of the reflected light intensities of the histogram for a second pixel among the plurality of pixels.

5. A distance measuring device according to claim 4.

10. an output unit that outputs distance information regarding a distance to the object based on a first distance, which is a distance to the object acquired by the first distance measuring unit, and a second distance, which is a distance to the object acquired by the second distance measuring unit; 2. The distance measuring device according to claim 1, further comprising:

11. The output unit is If the first distance or the second distance is longer than a predetermined threshold, output the first distance as the distance information; When the first distance and the second distance are equal to or smaller than a predetermined threshold, the second distance is output as the distance information.

11. The distance measuring device according to claim 10.

12. The output unit is When the reflected light from the object is detected, the first distance is output as the distance information; If the reflected light from the object is not detected, the second distance is output as the distance information.

11. The distance measuring device according to claim 10.

13. The output unit is When the distance to the object can be obtained from the difference between the outputs of the plurality of pixels, the second distance is output as the distance information; When the distance to the object cannot be obtained from the difference between the outputs of the plurality of pixels, the first distance is output as the distance information.

11. The distance measuring device according to claim 10.

14. Each of the pixels includes an avalanche photodiode.

2. A distance measuring device according to claim 1.

15. A distance measuring device according to any one of claims 1 to 14, a signal processing unit that processes a signal output by the distance measuring device; A ranging system comprising:

16. A moving object comprising the distance measuring device according to any one of claims 1 to 14, A control unit controls the movement of the moving body using information on the distance to the object acquired by the distance measuring device. A moving object characterized by:

17. A distance measuring device according to any one of claims 1 to 14, an optical device corresponding to the distance measuring device; A control device for controlling the distance measuring device; a processing device for processing a signal output from the distance measuring device; a display device for displaying information obtained by the distance measuring device; a storage device for storing information obtained by the distance measuring device; and a mechanical device that operates based on the information obtained by the distance measuring device; At least one of An apparatus comprising:

Images