Photoelectric conversion device, photoelectric conversion system

Abstract

To solve a problem of a trade-off relation between distance resolution and a frame rate in a distance measuring method of a Time-gate system.SOLUTION: A photoelectric conversion device includes: a pixel array in which a plurality of pixels each including a photoelectric conversion element is arranged; a first micro frame reading unit for acquiring a first micro frame; a second micro frame reading unit for acquiring a second micro frame; a first micro frame addition unit for synthesizing the first micro frame to generate a first sub frame; and a second micro frame addition unit for synthesizing the second micro frame to generate a second sub frame. A first exposure period in which a signal constituting the first micro frame is generated, is controlled by a first control signal and a second exposure period in which a signal constituting the second micro frame is generated, is controlled by a second control signal. In one micro frame acquisition period, the first exposure period differs from the second exposure period.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device and a photoelectric conversion system including the photoelectric conversion device. [Background technology]

[0002] Patent Document 1 discloses a distance measuring device that measures the distance to an object by emitting light from a light source and receiving light including reflected light from the object with a light receiving element. Patent Document 1 discloses a method of repeatedly performing measurements while changing a gating period during which photons are detected in a SPAD (Single Photon Avalanche Diode) element. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2017 / 0052065 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method described in Patent Document 1 (the time-gate method), there is a trade-off between distance resolution and frame rate. [Means for solving the problem]

[0005] One aspect of the present invention is a photoelectric conversion device having a pixel array in which a plurality of pixels having photoelectric conversion elements are arranged in an array, a first microframe reading unit that acquires a first microframe based on incident light to each of the plurality of photoelectric conversion elements, a second microframe reading unit that acquires a second microframe based on incident light to each of the plurality of photoelectric conversion elements, a first microframe adder unit that generates a first subframe by combining the first microframes, and a second microframe adder unit that generates a second subframe by combining the second microframes, wherein a first exposure period during which a signal constituting the first microframe is generated is controlled by a first control signal, and a second exposure period during which a signal constituting the second microframe is generated is controlled by a second control signal, and the first exposure period and the second exposure period are different from each other during the acquisition period of one microframe.

[0006] Another aspect of the present invention is a photoelectric conversion device comprising: a pixel array in which a plurality of pixels, each having an avalanche photodiode, are arranged in an array; an input terminal of a first gating circuit connected in parallel to an output terminal of the avalanche photodiode and receiving a signal constituting a first microframe; an input terminal of a second gating circuit connected in parallel to an output terminal of the avalanche photodiode and receiving a signal constituting a second microframe; a first microframe readout unit connected to the output terminal of the first gating circuit via a first counter circuit that counts the signals constituting the first microframe and acquiring the first microframe; a second microframe readout unit connected to the output terminal of the second gating circuit via a second counter circuit that counts the signals constituting the second microframe and acquiring the second microframe; a first microframe adder unit that generates a first subframe by combining the first microframes; and a second microframe adder unit that generates a second subframe by combining the second microframes. Effect of the Invention

[0007] According to the present invention, it is possible to improve the frame rate while ensuring distance resolution. [Brief description of the drawings]

[0008] [Figure 1] 1 is a hardware block diagram showing an example of a schematic configuration of a distance measuring device according to a first embodiment. [Diagram 2] 1 is a schematic diagram showing the overall configuration of a photoelectric conversion device according to a first embodiment. [Diagram 3] 1 is a schematic block diagram showing an example of the configuration of a sensor substrate according to a first embodiment. [Figure 4] 1 is a schematic block diagram showing an example of the configuration of a circuit board according to a first embodiment. [Diagram 5] 2 is a schematic block diagram showing an example of the configuration of one pixel of a photoelectric conversion unit and a pixel signal processing unit according to the first embodiment. FIG. [Figure 6] 3A to 3C are diagrams illustrating the operation of the avalanche photodiode according to the first embodiment. [Figure 7] 1 is a functional block diagram showing a schematic configuration example of a distance measuring device according to a first embodiment. [Figure 8] 2A to 2C are schematic diagrams for explaining ranging frames, subframes, and microframes according to the first embodiment. [Figure 9] 5 is a flowchart showing the operation of the distance image generating device according to the first embodiment in one ranging frame period. [Figure 10] FIG. 4 is a drive timing chart according to the first embodiment. [Figure 11] 10 is a schematic block diagram showing an example of the configuration of a circuit board 21 according to a second embodiment. FIG. [Figure 12] FIG. 11 is a schematic block diagram showing an example of the configuration of a circuit board 21 according to a third embodiment. [Figure 13] 13A and 13B are schematic diagrams and drive timing charts of a pixel array according to a fourth embodiment. [Figure 14] FIG. 13 is a schematic block diagram of a pixel according to a fifth embodiment. [Figure 15] FIG. 13 is a drive timing chart according to the fifth embodiment. [Figure 16] FIG. 13 is a drive timing chart according to the sixth embodiment. [Figure 17] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a seventh embodiment. [Figure 18] FIG. 13 is a functional block diagram of a photoelectric conversion system according to an eighth embodiment. [Figure 19] FIG. 13 is a functional block diagram of a photoelectric conversion system according to a ninth embodiment. [Figure 20] FIG. 23 is a functional block diagram of a photoelectric conversion system according to a tenth embodiment. [Figure 21] FIG. 23 is a functional block diagram of a photoelectric conversion system according to an eleventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) will be used as necessary. The use of these terms is for the purpose of facilitating understanding of the embodiment with reference to the drawings, and the technical scope of the present invention is not limited by the meaning of these terms.

[0011] In this specification, the plan view refers to a view from a direction perpendicular to the light incident surface of the semiconductor layer. Also, the cross-sectional view refers to a surface in a direction perpendicular to the light incident surface of the semiconductor layer. Note that, when the light incident surface of the semiconductor layer is a rough surface when viewed microscopically, the plan view is defined based on the light incident surface of the semiconductor layer when viewed macroscopically.

[0012] The semiconductor layer has a first surface and a second surface, which is the surface opposite to the first surface and on which light is incident. In this specification, the depth direction is the direction from the first surface to the second surface of the semiconductor layer on which the APD is arranged. Hereinafter, the "first surface" may be referred to as the "front surface" and the "second surface" may be referred to as the "rear surface." The "depth" of a point or region in the semiconductor layer means the distance from the first surface (front surface) of the point or region. When there is a point (or region) Z1 whose distance (depth) from the first surface is d1 and a point (or region) Z2 whose distance (depth) from the first surface is d2, and d1>d2, it may be expressed as "Z1 is deeper than Z2" or "Z2 is shallower than Z1." Also, when there is a point (or region) Z3 whose distance (depth) from the first surface is d3, and d1>d3>d2 holds, it can be expressed as "Z3 is at a depth between Z1 and Z2" or "Z3 is between Z1 and Z2 in the depth direction."

[0013] In this specification, when the term "impurity concentration" is used simply, it means the net impurity concentration minus the amount compensated by the impurity of the opposite conductivity type. In other words, "impurity concentration" refers to the NET doping concentration. A region where the P-type doped impurity concentration is higher than the N-type doped impurity concentration is a P-type semiconductor region. Conversely, a region where the N-type doped impurity concentration is higher than the P-type doped impurity concentration is an N-type semiconductor region.

[0014] In the following embodiments, the connection between elements of a circuit may be described. In this case, even if another element is interposed between the elements of interest, the elements of interest are treated as being connected to each other unless otherwise specified. For example, assume that element A is connected to one node of a capacitive element C having multiple nodes, and element B is connected to the other node. Even in such a case, element A and element B are treated as being connected to each other unless otherwise specified.

[0015] <First embodiment> 1 is a hardware block diagram showing an example of the schematic configuration of a distance image generating device 30 according to this embodiment. The distance image generating device 30 has a light emitting device 31, a light receiving device 32, and a signal processing circuit 33. Note that the configuration of the distance image generating device 30 shown in this embodiment is just an example, and is not limited to the configuration shown in the figure.

[0016] The distance image generating device 30 is a device that measures the distance from the distance image generating device 30 to the object X to be measured using technology such as LiDAR (Light Detection And Ranging). The distance image generating device 30 measures the distance from the distance image generating device 30 to the object X based on the time difference between when light is emitted from the light emitting device 31 and when the emitted light is reflected by the object X and received by the light receiving device 32. The distance image generating device 30 can also measure the distance to a plurality of points two-dimensionally by emitting laser light to a predetermined distance range including the object X and receiving the reflected light by a pixel array. This allows the distance image generating device 30 to generate and output a distance image.

[0017] The light received by the light receiving device 32 includes ambient light such as sunlight in addition to the reflected light from the object X. Therefore, the distance image generating device 30 performs distance measurement with reduced influence of ambient light by using a method of measuring the light incident in each of a plurality of periods (bin periods) and determining that the reflected light is incident in the bin period in which the amount of light is at its peak.

[0018] The light emitting device 31 is a device that emits light such as laser light to the outside of the distance image generating device 30. The signal processing circuit 33 may include a processor that performs arithmetic processing of digital signals, a memory that stores digital signals, etc. The memory may be, for example, a semiconductor memory such as an SRAM or a DRAM.

[0019] The light receiving device 32 generates a pulse signal including a pulse based on the incident light. The light receiving device 32 is, for example, a photoelectric conversion device including an avalanche photodiode (hereinafter, APD) as a photoelectric conversion element. In this case, when one photon is incident on the APD and an electric charge is generated, avalanche multiplication occurs, and one pulse is generated according to the avalanche current. However, the light receiving device 32 may be one using, for example, a photoelectric conversion element using another photodiode.

[0020] In this embodiment, the light receiving device 32 includes a pixel array in which pixels including photoelectric conversion elements are arranged in an array of multiple rows and multiple columns. Here, a photoelectric conversion device as a specific configuration example of the light receiving device 32 will be described with reference to Fig. 2 to Fig. 6. The configuration example of the photoelectric conversion device described below is one example. The photoelectric conversion device applicable to the light receiving device 32 is not limited to this, and may be any device capable of realizing the function of Fig. 7 described later.

[0021] FIG. 2 is a schematic diagram showing the overall configuration of a photoelectric conversion device 100 according to this embodiment. The photoelectric conversion device 100 has a sensor substrate 11 (first substrate) and a circuit substrate 21 (second substrate) that are stacked on each other. The sensor substrate 11 and the circuit substrate 21 are electrically connected to each other. The sensor substrate 11 has a pixel region 12 in which a plurality of pixels 101 are arranged to form a plurality of rows and a plurality of columns. The circuit substrate 21 has a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns, and a second circuit region 23 arranged on the periphery of the first circuit region 22. The second circuit region 23 may include a circuit for controlling the plurality of pixel signal processing units 103. The sensor substrate 11 has a light incident surface that receives incident light and a connection surface that faces the light incident surface. The sensor substrate 11 is connected to the circuit substrate 21 on the connection surface side. That is, the photoelectric conversion device 100 is a so-called back-illuminated photoelectric conversion device.

[0022] In the following description, the sensor substrate 11 and the circuit substrate 21 are described as being diced chips, but the sensor substrate 11 and the circuit substrate 21 are not limited to being chips. For example, the sensor substrate 11 and the circuit substrate 21 may be wafers. In addition, when the sensor substrate 11 and the circuit substrate 21 are diced chips, the photoelectric conversion device 100 may be manufactured by stacking them in a wafer state and then dicing them, or may be manufactured by stacking them after dicing.

[0023] 3 is a schematic block diagram showing an example of the arrangement of the sensor substrate 11. A plurality of pixels 101 arranged in a plurality of rows and a plurality of columns are arranged in the pixel region 12. Each of the plurality of pixels 101 has a photoelectric conversion unit 102 including an APD as a photoelectric conversion element within the substrate.

[0024] The conductivity type of the charge pair generated in the APD and used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type in which the charge of the same polarity as the signal charge is the majority carrier. The conductivity type opposite to the first conductivity type, that is, the conductivity type in which the charge of the opposite polarity to the signal charge is the majority carrier, is called the second conductivity type. In the following description, the anode of the APD is set to a fixed potential, and a signal is taken out from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is an N-type semiconductor region, and the semiconductor region of the second conductivity type is a P-type semiconductor region. The cathode of the APD may be set to a fixed potential, and the signal may be taken out from the anode of the APD. In this case, the semiconductor region of the first conductivity type is a P-type semiconductor region, and the semiconductor region of the second conductivity type is an N-type semiconductor region. In the following, a case in which one node of the APD is set to a fixed potential will be described, but the potentials of both nodes may be fluctuated.

[0025] 4 is a schematic block diagram showing a configuration example of the circuit board 21. The circuit board 21 has a first circuit area 22 in which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns.

[0026] Further, the circuit board 21 is provided with a first vertical scanning circuit 110, a first horizontal scanning circuit 111, a first readout circuit 112, a first pixel output signal line 113, a first output circuit 114, and a first control signal generating unit 115. In addition, the circuit board 21 is provided with a second vertical scanning circuit 116, a second horizontal scanning circuit 117, a second readout circuit 118, a second pixel output signal line 119, a second output circuit 120, and a second control signal generating unit 121. The multiple photoelectric conversion units 102 shown in FIG. 3 and the multiple pixel signal processing units 103 shown in FIG. 4 are electrically connected to each other via connection wiring provided for each pixel 101. In this embodiment, the pixel signal processing unit 103 is configured to be able to output two types of signals (first signal, second signal) having different exposure periods from the pixel 101.

[0027] The first control signal generating unit 115 is a control circuit that generates control signals for driving the first vertical scanning circuit 110, the first horizontal scanning circuit 111, and the first readout circuit 112, and supplies them to each of these units. As a result, the first control signal generating unit 115 controls the drive timing of each unit, etc. In addition, the second control signal generating unit 121 is a control circuit that generates control signals for driving the first vertical scanning circuit 116, the first horizontal scanning circuit 117, and the first readout circuit 118, and supplies them to each of these units. As a result, the first control signal generating unit 121 controls the drive timing of each unit, etc.

[0028] The first vertical scanning circuit 110 supplies a control signal to each of the pixel signal processing units 103 based on the control signal supplied from the first control signal generating unit 115. The first vertical scanning circuit 110 supplies a control signal to each pixel signal processing unit 103 for each row via a first drive line provided for each row of the first circuit area 22. As described later, there may be multiple drive lines for each row. The first vertical scanning circuit 110 may include logic circuits such as a shift register and an address decoder. In this way, the first vertical scanning circuit 110 selects a row for outputting a first signal from the pixel signal processing unit 103.

[0029] The second vertical scanning circuit 116 supplies a control signal to each of the pixel signal processing units 103 based on the control signal supplied from the second control signal generation unit 121. The second vertical scanning circuit 116 supplies a control signal to each pixel signal processing unit 103 for each row via a second drive line provided for each row of the first circuit area 22. As described later, there may be multiple drive lines for each row. The second vertical scanning circuit 116 may include logic circuits such as a shift register and an address decoder. In this way, the second vertical scanning circuit 116 selects a row for outputting the second signal from the pixel signal processing unit 103.

[0030] The signal output from the photoelectric conversion unit 102 of the pixel 101 is processed by the pixel signal processing unit 103. The pixel signal processing unit 103 counts the number of pulses output from the APD included in the photoelectric conversion unit 102 to obtain and hold a digital signal having multiple bits.

[0031] The first horizontal scanning circuit 111 supplies a control signal to the first readout circuit 112 based on the control signal supplied from the first control signal generating unit 115. The pixel signal processing unit 103 is connected to the first readout circuit 112 via a first pixel output signal line 113 provided for each column of the first circuit region 22. The first pixel output signal line 113 of one column is shared by a plurality of pixel signal processing units 103 of the corresponding column. The first pixel output signal line 113 includes a plurality of wirings. The first pixel output signal line 113 has at least a function of outputting a digital signal from each pixel signal processing unit 103 to the first readout circuit 112 and a function of supplying a control signal for selecting a column to output a signal to the pixel signal processing unit 103. The first readout circuit 112 outputs a signal to a storage unit or a signal processing unit outside the photoelectric conversion device 100 via the first output circuit 114 based on the control signal supplied from the first control signal generating unit 115.

[0032] The second horizontal scanning circuit 117 supplies a control signal to the second readout circuit 118 based on the control signal supplied from the second control signal generation unit 121. The pixel signal processing unit 103 is connected to the second readout circuit 118 via a second pixel output signal line 119 provided for each column of the first circuit region 22. The second pixel output signal line 119 of one column is shared by a plurality of pixel signal processing units 103 of the corresponding column. The second pixel output signal line 119 includes a plurality of wirings. The second pixel output signal line 119 has at least a function of outputting a digital signal from each pixel signal processing unit 103 to the second readout circuit 118 and a function of supplying a control signal for selecting a column to output a signal to the pixel signal processing unit 103. The second readout circuit 118 outputs a signal to a storage unit or a signal processing unit outside the photoelectric conversion device 100 via the second output circuit 120 based on the control signal supplied from the second control signal generation unit 121.

[0033] The photoelectric conversion units 102 in the pixel region 12 may be arranged one-dimensionally. Moreover, it is not necessary to provide one pixel signal processing unit 103 for each pixel 101. For example, one pixel signal processing unit 103 may be shared by a plurality of pixels 101. In this case, the pixel signal processing unit 103 provides a signal processing function to each pixel 101 by sequentially processing signals output from each photoelectric conversion unit 102.

[0034] 3 and 4, a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged is arranged in a region overlapping the pixel region 12 in a plan view. A first vertical scanning circuit 110, a first horizontal scanning circuit 111, a first readout circuit 112, a first output circuit 114, a first control signal generating unit 115, and the like are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a plan view. Similarly, a second vertical scanning circuit 116, a second horizontal scanning circuit 117, a second readout circuit 118, a second output circuit 120, and a second control signal generating unit 121 are also arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. In the circuit board 21, a second circuit area 23 (described above in FIG. 2) is arranged in an area overlapping the non-pixel area in a planar view, in which a first vertical scanning circuit 110, a first horizontal scanning circuit 111, a first readout circuit 112, a first output circuit 114, a first control signal generating unit 115, a second vertical scanning circuit 116, a second horizontal scanning circuit 117, a second readout circuit 118, a second output circuit 120, and a second control signal generating unit 121 are arranged.

[0035] The arrangement of the first pixel output signal lines 113, the arrangement of the first readout circuits 112, and the arrangement of the first output circuits 114 are not limited to those shown in Fig. 4. For example, the first pixel output signal lines 113 may be arranged to extend in the row direction and shared by a plurality of pixel signal processing units 103 in the corresponding row. The first readout circuits 112 may be arranged so that the first pixel output signal lines 113 in each row are connected to each other.

[0036] Similarly, the arrangement of the second pixel output signal lines 119, the arrangement of the second readout circuits 118, and the arrangement of the second output circuits 120 are not limited to those shown in Fig. 4. For example, the second pixel output signal lines 119 may be arranged to extend in the row direction and shared by a plurality of pixel signal processing units 103 in the corresponding row. The second readout circuits 118 may be arranged so that the second pixel output signal lines 119 in each row are connected to each other.

[0037] 5 is a schematic block diagram showing a configuration example of one pixel of the photoelectric conversion unit 102 and pixel signal processing unit 103 according to this embodiment. FIG. 5 shows a more specific configuration example including a connection relationship between the photoelectric conversion unit 102 arranged on the sensor substrate 11 and the pixel signal processing unit 103 arranged on the circuit substrate 21. In FIG. 5, the drive lines between the first vertical scanning circuit 110 and the pixel signal processing unit 103 in FIG. 4 are shown as drive lines 213, 214, and 215. Similarly, the drive lines between the second vertical scanning circuit 116 and the pixel signal processing unit 103 are shown as drive lines 219, 220, and 221.

[0038] The photoelectric conversion unit 102 has an APD 201. The pixel signal processing unit 103 has a quenching element 202, a waveform shaping unit 210, a first counter circuit 211, a first selection circuit 212, and a first gating circuit 216. The pixel signal processing unit 103 further has a second counter circuit 217, a second selection circuit 218, and a second gating circuit 222.

[0039] The APD 201 generates charges according to incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A cathode of the APD 201 is connected to a first terminal of the quench element 202 and an input terminal of the waveform shaping unit 210. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD 201 such that the APD 201 performs avalanche multiplication. When charges are generated by incident light in the APD 201 to which the reverse bias voltage is supplied, the charges undergo avalanche multiplication, generating an avalanche current.

[0040] In addition, there are two operation modes when a reverse bias voltage is supplied to the APD 201: Geiger mode and linear mode. The Geiger mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is greater than the breakdown voltage, and the linear mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is close to or less than the breakdown voltage.

[0041] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD 201 may be operated in either linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger than that of an APD in linear mode, and the effect of avalanche multiplication is more pronounced, so a SPAD is preferable.

[0042] The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication. The quench element 202 suppresses the voltage supplied to the APD 201 to suppress avalanche multiplication (quench operation). The quench element 202 also returns the voltage supplied to the APD 201 to the voltage VH by passing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 202 can be, for example, a resistive element.

[0043] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 5, the waveform shaping unit 210 may be a circuit in which a plurality of inverters are connected in series, or may be another circuit having a waveform shaping effect.

[0044] The first gating circuit 216 and the second gating circuit 222 are circuits that perform gating to pass a pulse signal output from the waveform shaping unit 210 for a predetermined period. An input terminal of the first gating circuit 216 and an input terminal of the second gating circuit 222 are connected in parallel to an output terminal of the APD 201. During a period in which a pulse signal can pass through the first gating circuit 216, photons that have entered the APD 201 are counted by a subsequent first counter circuit 211. Similarly, during a period in which a pulse signal can pass through the second gating circuit 222, photons that have entered the APD 201 are counted by a subsequent second counter circuit 217. Thus, the first gating circuit 216 and the second gating circuit 222 control a period in which a signal is generated based on incident light in the pixel 101. The period during which the pulse signal can pass through each gating circuit is controlled by a control signal supplied from the first vertical scanning circuit 110 via a drive line 215 and a control signal supplied from the second vertical scanning circuit 116 via a drive line 221. FIG. 5 shows an example in which one AND circuit is used as each of the first gating circuit 216 and the second gating circuit 222. A pulse signal is input to one of two input terminals of the AND circuit constituting the first gating circuit 216, and a control signal supplied from the first vertical scanning circuit 110 via the drive line 215 is input to the other. A pulse signal is input to one of two input terminals of the AND circuit constituting the second gating circuit 222, and a control signal supplied from the second vertical scanning circuit 116 via the drive line 221 is input to the other. The AND circuit serving as the first gating circuit 216 outputs the logical product thereof to the first counter circuit 211, and the AND circuit serving as the second gating circuit 222 outputs the logical product thereof to the second counter circuit 217. Note that the first gating circuit 216 and the second gating circuit 222 may have a circuit configuration other than an AND circuit as long as they can realize gating.

[0045] The first counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 via the first gating circuit 216, and holds a digital signal indicating the count value. When a control signal is supplied from the first vertical scanning circuit 110 via the drive line 213, the first counter circuit 211 resets the signal it holds.

[0046] The second counter circuit 217 counts the pulse signal output from the waveform shaping unit 210 via the second gating circuit 222, and holds a digital signal indicating the count value. When a control signal is supplied from the second vertical scanning circuit 116 via the drive line 219, the second counter circuit 217 resets the signal it holds.

[0047] A control signal is supplied to the first selection circuit 212 from the first vertical scanning circuit 110 shown in Fig. 4 via a drive line 214 shown in Fig. 5. In response to this control signal, the first selection circuit 212 switches between electrical connection and non-connection between the first counter circuit 211 and the first pixel output signal line 113. The first selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to a value held in the first counter circuit 211.

[0048] A control signal is supplied to the second selection circuit 218 from the second vertical scanning circuit 116 shown in Fig. 4 via a drive line 220 shown in Fig. 5. In response to this control signal, the second selection circuit 218 switches between electrical connection and non-connection between the second counter circuit 217 and the second pixel output signal line 119. The second selection circuit 218 includes, for example, a buffer circuit for outputting a signal corresponding to a value held in the second counter circuit 217.

[0049] In the example of FIG. 5, the first selection circuit 212 and the second selection circuit 218 switch between electrical connection and disconnection between the first counter circuit 211 and the first pixel output signal line 113, and between the second counter circuit 217 and the second pixel output signal line 119. However, the method of controlling the signal output to the first pixel output signal line 113 and the second pixel output signal line 119 is not limited to this. For example, a switch such as a transistor may be disposed at a node between the quench element 202 and the APD 201, between the photoelectric conversion unit 102 and the pixel signal processing unit 103, etc. The signal output to the first pixel output signal line 113 and the second pixel output signal line 119 may be controlled by switching between electrical connection and disconnection of these switches. In addition, the signal output to the first pixel output signal line 113 and the second pixel output signal line 119 may be controlled by changing the value of the voltage VH or voltage VL supplied to the photoelectric conversion unit 102 using a switch such as a transistor.

[0050] Figures 6(a), 6(b) and 6(c) are diagrams for explaining the operation of the APD 201 according to this embodiment. Figure 6(a) is a diagram showing the APD 201, the quench element 202 and the waveform shaping unit 210 extracted from Figure 5. As shown in Figure 6(a), the connection node of the APD 201, the quench element 202 and the input terminals of the waveform shaping unit 210 is referred to as nodeA. Also, as shown in Figure 6(a), the output side of the waveform shaping unit 210 is referred to as nodeB.

[0051] FIG. 6(b) is a graph showing the time change of the potential of nodeA in FIG. 6(a). FIG. 6(c) is a graph showing the time change of the potential of nodeB in FIG. 6(a). In the period from time t0 to time t1, a voltage of VH-VL is applied to the APD 201 in FIG. 6(a). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201. As a result, an avalanche current flows through the quench element 202, and the potential of nodeA drops. After that, the amount of potential drop further increases, and the voltage applied to the APD 201 gradually decreases. Then, at time t2, the avalanche multiplication in the APD 201 stops. As a result, the voltage level of nodeA does not drop below a certain value. After that, in the period from time t2 to time t3, a current that compensates for the voltage drop flows from the node of voltage VH to nodeA, and at time t3, nodeA is stabilized to its original potential.

[0052] In the above process, the potential of node B becomes high during the period when the potential of node A is lower than a certain threshold. In this way, the waveform of the drop in the potential of node A caused by the incidence of a photon is shaped by the waveform shaping unit 210 and output as a pulse to node B.

[0053] Next, the overall configuration and operation of the distance image generating device 30 will be described in more detail. Fig. 7 is a functional block diagram showing an example of a schematic configuration of the distance image generating device 30 according to this embodiment. Fig. 7 shows a more detailed configuration of the light emitting device 31, the light receiving device 32, and the signal processing circuit 33 described in Fig. 1.

[0054] The light emitting device 31 has a pulsed light source 311 and a light source control unit 312. The pulsed light source 311 is a light source such as a semiconductor laser device that emits pulsed light throughout the entire distance measurement range. The pulsed light source 311 may be a surface light source such as a surface emitting laser. The light source control unit 312 is a control circuit that controls the light emission timing of the pulsed light source 311.

[0055] The light receiving device 32 has an imaging section 321, a gate pulse generating section 322, a first microframe readout section 323, a first microframe adder section 324, an addition count control section 325, and a first subframe output section 326. The light receiving device 32 further has a second microframe readout section 327, a second microframe adder section 328, and a second subframe output section 329. As described above, the imaging section 321 may be a photoelectric conversion device including a pixel array in which pixel circuits including the APD 201 are arranged two-dimensionally. This allows the distance image generating device 30 to obtain a two-dimensional distance image.

[0056] The gate pulse generating unit 322 is a control circuit that outputs a control signal that controls the drive timing of the imaging unit 321. The gate pulse generating unit 322 transmits and receives a control signal to and from the light source control unit 312 to synchronously control the pulse light source 311 and the imaging unit 321. This enables imaging with a controlled time difference between the time when light is emitted from the pulse light source 311 and the time when light is received by the imaging unit 321. In this embodiment, the gate pulse generating unit 322 drives the imaging unit 321 with a global gate. The global gate drive is a drive method in which incident light is detected simultaneously in several pixels (pixel groups) in the imaging unit 321 during the same exposure period, based on the emission time of the pulse light from the pulse light source 311. In the global gate drive of this embodiment, the detection of incident light is repeatedly performed while sequentially shifting the timing from the emission of light to the collective exposure. As a result, each pixel of the imaging unit 321 simultaneously generates a 1-bit signal indicating the presence or absence of incident photons during each of a plurality of exposure periods. The generated 1-bit signal may be held in the first counter circuit 211 and the second counter circuit 217. In this case, the first counter circuit 211 and the second counter circuit 217 are configured as 1-bit counters (memory circuits). Hereinafter, the first counter circuit 211 will be referred to as the first memory circuit 211, and the second counter circuit 217 will be referred to as the second memory circuit 217.

[0057] This global gate drive is realized by inputting a high-level signal to the input terminals of the first gating circuit 216 and the second gating circuit 222 of the multiple pixels 101 during a gating period based on a control signal from the gate pulse generating unit 322. Note that it is assumed that the period during which the signal input to the first gating circuit 216 is at a high level is different from the period during which the signal input to the second gating circuit 222 is at a high level, but the periods may overlap in part or in whole. Also, the period during which the signal input to the first gating circuit 216 is at a high level may be exactly the same as the period during which the signal input to the second gating circuit 222 is at a high level.

[0058] The first microframe readout unit 323, the first microframe adder 324, the addition count control unit 325, the second microframe readout unit 327, and the second microframe adder 328 read out a 1-bit signal constituting a microframe from the imaging unit 321. These circuits are signal processing circuits that perform predetermined signal processing on the read out 1-bit signal. Details of the operation of each of these units will be described later with reference to FIG. 9. The first subframe output unit 326 and the second subframe output unit 329 are interfaces that output a signal from the light receiving device 32 to the signal processing circuit 33 in accordance with a predetermined standard. The first subframe output unit 326 and the second subframe output unit 329 transmit a signal from the memory in the light receiving device 32 to the memory in the signal processing circuit 33.

[0059] Signal processing circuit 33 has a first subframe group storage unit 331, a second subframe group storage unit 333, and a distance image generation unit 332. Signal processing circuit 33 is a computer including a processor that operates as distance image generation unit 332, and a memory or the like that operates as first subframe group storage unit 331 and second subframe group storage unit 333. The operation of each of these units will also be described later with reference to FIG. 9.

[0060] Next, prior to describing the driving method of this embodiment, the configurations of ranging frames, subframes, and microframes will be described with reference to Fig. 8. Fig. 8 shows a schematic diagram of acquisition periods of ranging frames corresponding to distance images, subframes used to generate ranging frames, and microframes used to generate subframes, arranged in blocks in the horizontal direction. The horizontal direction in Fig. 8 indicates the passage of time, and one block indicates the acquisition period of one ranging frame, subframe, or microframe.

[0061] The ranging frame F1 corresponds to one distance image. That is, the ranging frame F1 has information corresponding to the distance to the object X, calculated from the time difference between when light is emitted and when it is received, for each of a plurality of pixels. In this embodiment, it is assumed that the distance image is acquired as a video, and acquisition of one ranging frame F1 is repeated every time one ranging frame period T1 elapses.

[0062] One ranging frame F1 is generated from multiple subframes F2. One ranging frame period T1 includes multiple subframe periods T2. One subframe F2 is repeatedly acquired every time one subframe period T2 elapses. The subframe F2 is composed of a multi-bit signal that corresponds to the amount of light incident in the subframe period T2.

[0063] One subframe F2 is generated from multiple microframes F3. One subframe period T2 includes multiple microframe periods T3. One microframe F3 is repeatedly acquired every time one microframe period T3 elapses. The microframe F3 is composed of a 1-bit signal indicating the presence or absence of incident light to the photoelectric conversion element in the microframe period T3. One subframe F2 of a multi-bit signal is generated by adding and combining multiple microframes of 1-bit signals. As a result, one subframe F2 can include a multi-bit signal corresponding to the number of microframes in which incident light is detected within the subframe period T2.

[0064] In this way, a plurality of subframes F2 in which the period for acquiring the incident light differs from each other are acquired. This signal acquisition time can be associated with the distance from the distance image generating device to the distance measurement target. Then, the signal acquisition time at which the signal value is maximum can be determined from the distribution of the signal acquisition time and the signal values ​​of the plurality of subframes F2. Since it is estimated that the reflected light is incident on the imaging section 321 at the time when the signal value is maximum, the distance can be calculated by converting the signal acquisition time at which the signal value is maximum into the distance to the target X. Also, a distance image can be generated by calculating the distance for each pixel and acquiring a two-dimensional distribution of the distance.

[0065] 8, the length of a ranging frame period T1 required to acquire one ranging frame F1 depends on the number of subframes F2. Since the number of subframes F2 is a parameter corresponding to the number of ranging points, there is a trade-off between the distance resolution and the frame rate.

[0066] 9 shows a method for driving the distance image generating device in one distance measurement frame period T1. The driving method of this embodiment will be described with reference to the flowchart of FIG.

[0067] In the flowchart shown in Fig. 9, the processing from "START" to "END" indicates processing performed in a ranging frame period T1 in which one ranging frame F1 in Fig. 8 is acquired. One cycle of processing in the loop from step S11 to step S15 indicates processing performed in a sub-frame period T2 in which one sub-frame F2 in Fig. 8 is acquired. One cycle of processing in the loop from step S11 to step S13 indicates processing performed in a micro-frame period T3 in which one micro-frame F3 in Fig. 8 is acquired.

[0068] In step S11, the light source control unit 312 controls the pulse light source 311 to emit pulse light within a predetermined distance measurement range. In synchronization with this, the gate pulse generation unit 322 controls the imaging unit 321 to start detecting incident light by global gate driving. Here, the gate pulse generation unit 322 generates gate pulses indicating different detection periods for the first gating circuit 216 and the second gating circuit 222.

[0069] In step S12, the first microframe readout section 323 and the second microframe readout section 327 each read out a microframe from the imaging section 321 every time a microframe period elapses. The first microframe readout section 323 reads out a 1-bit signal constituting the first microframe stored in the first memory circuit 211. The second microframe readout section 327 reads out a 1-bit signal constituting the second microframe stored in the second memory circuit 217. The read out microframes are stored in the memories of the first microframe adder section 324 and the second microframe adder section 328. This memory has a storage capacity capable of storing multiple bits of data for each pixel. The first microframe adder section 324 and the second microframe adder section 328 sequentially add the value of the microframe to the value stored in the memory every time a microframe is read out. As a result, the first microframe adder 324 and the second microframe adder 328 add a plurality of microframes in a subframe period to generate a subframe. The number of additions in the first microframe adder 324 and the second microframe adder 328 is controlled by the addition number control unit 325. For this control, the addition number control unit 325 holds information on the number of additions set in advance. In this way, the first microframe readout unit 323 and the second microframe readout unit 327 function as an acquisition unit that acquires a microframe composed of a 1-bit signal based on the light incident on the photoelectric conversion element. In addition, the first microframe adder 324 and the second microframe adder 328 function as a synthesis unit that synthesizes a plurality of microframes acquired in different periods. For example, when the number of additions is 64, a subframe signal having a 6-bit gradation can be generated by synthesizing 64 microframes.

[0070] In step S13, the first microframe adder 324 and the second microframe adder 328 determine whether or not a preset number of microframes have been added. If the preset number of microframes have not been added (NO in step S13), the process proceeds to step S11, where the next microframe is read. If the preset number of microframes have been added (YES in step S13), the process proceeds to step S14.

[0071] In step S14, the first subframe output unit 326 reads the subframe for which the addition has been completed from the memory of the first microframe adder 324, and outputs it to the first subframe group storage unit 331. Similarly, the second subframe output unit 329 reads the subframe for which the addition has been completed from the memory of the second microframe adder 328, and outputs it to the second subframe group storage unit 333. The first subframe group storage unit 331 stores the subframe output from the first subframe output unit 326. The second subframe group storage unit 333 stores the subframe output from the second subframe output unit 329. The first subframe group storage unit 331 and the second subframe group storage unit 333 are configured to be able to store a plurality of subframes used to generate one ranging frame individually for each subframe period.

[0072] In step S15, the signal processing circuit 33 judges whether the first subframe group storage unit 331 and the second subframe group storage unit 333 have completed acquisition of a predetermined number of subframes (i.e., the number of ranging points). If acquisition of subframes for the number of ranging points has not been completed (NO in step S15), the process proceeds to step S11, and multiple microframes are acquired and added again to read the next subframe. In this case, the start time of global gate driving relative to the light emission time is shifted (gate shift) by one subframe period, and the same process is performed. If acquisition of subframes for the number of ranging points has been completed (YES in step S15), the process proceeds to step S16. Subframes for the number of ranging points are acquired by the loop between step S11 and step S15.

[0073] In step S16, the distance image generating unit 332 acquires a plurality of subframes in one ranging frame period from the first subframe group storage unit 331 and the second subframe group storage unit 333. The distance image generating unit 332 generates a distance image showing a two-dimensional distribution of distances by calculating the distance corresponding to the subframe in which the signal value is maximum for each pixel. The distance image generating unit 332 then outputs the distance image to a device external to the signal processing circuit 33. This distance image can be used, for example, to detect the surrounding environment of the vehicle. Note that the distance image generating unit 332 may store the distance image in a memory inside the distance image generating device.

[0074] In this embodiment, two microframes can be acquired for each pixel in the imaging section 321. The exposure periods (set in step S11 in FIG. 9) for reading out the two microframes are different from each other. A specific example will be described with reference to FIG. 10. FIG. 10 is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0075] 10 indicates the timing of light emission of the pulsed light source 311. As shown in FIG 10, the pulsed light source 311 emits light at a constant cycle under the control of the light source control unit 312.

[0076] "P_G1" and "P_G2" in Fig. 10 indicate the input timings of multiple types of gate pulses input from the gate pulse generating unit 322 to the imaging unit 321. "P_G1" corresponds to the control signal input to the first gating circuit 216 via the drive line 215 in Fig. 5. "P_G2" corresponds to the control signal input to the second gating circuit 222 via the drive line 221 in Fig. 5. The gate pulses synchronized with the light emission timing L01 are respectively the gate pulse G01 and the gate pulse G02. The gate pulse G01 and the gate pulse G02 go high after a predetermined time has elapsed since the light emission of the pulsed light source 311, but the periods during which they go high within one microframe period are different from each other. In other words, the gate pulse G01 goes to high level when a first time has elapsed since the emission of the pulsed light source 311 (first exposure period), and the gate pulse G02 goes to high level when a second time has elapsed since the emission of the pulsed light source 311 (second exposure period). The first time and the second time are different periods, and both are shorter than the length of a microframe period.

[0077] In this way, by differentiating the timing at which the gate pulse G01 and the gate pulse G02 become high, it is possible to vary the exposure period for reading out the two microframes acquired for each pixel in the imaging section 321. Therefore, it is possible to measure two types of distance measurement points within one microframe period.

[0078] As described above, one subframe is generated by adding up multiple microframes. That is, a signal for generating a subframe at a different ranging point is output for each pixel in the imaging unit 321. When one subframe is generated, at the start of the next subframe period, the timing of the gate pulse in each pixel is gate shifted by a predetermined time interval from the timing of the gate pulse in the current subframe period. By such gate shifting, a predetermined period corresponding to each subframe period can be set as an exposure period.

[0079] On the other hand, as shown in FIG. 10, when the periods during which the gate pulse G01 and the gate pulse G02 are at a high level are adjacent to each other, if a photon is detected during the period during which the gate pulse G01 is at a high level, "1" is set in the first memory circuit 211, but at the same time, "1" may also be set in the second memory circuit 217. In order to prevent such double counting, a latch circuit that holds the output of the first gating circuit 216 may be added to the pixel signal processing unit 103. The output of the latch circuit is input to the second gating circuit 222, and when the output of the first gating circuit 216 becomes "1" and "1" is set in the first memory circuit 211, the latch circuit outputs "0" to mask the second gating circuit 222. This temporarily disables the second memory circuit 217, thereby preventing double counting.

[0080] In this embodiment, the pulsed light source 311 is configured to emit light of an infrared wavelength, and "emission" in Fig. 10 may be the timing of emission of light of an infrared wavelength by the pulsed light source 311. In this case, since visible light becomes noise in distance measurement, an infrared bandpass filter (visible light cut filter) may be attached. Also, a color filter that transmits only infrared light may be arranged on all pixels.

[0081] In this embodiment, the exposure period for reading out the two microframes acquired in each pixel is made different, and gate shift is performed for each subframe period for each pixel. This reduces the number of gate shifts required to measure the required number of ranging points, compared to the case where one microframe is acquired in each pixel. This shortens the total period of the multiple subframes, that is, the length of the frame period (T1 in FIG. 8). Therefore, the frame rate can be improved without reducing the number of ranging points (distance resolution). As described above, according to this embodiment, a photoelectric conversion device is provided in which the frame rate is improved while maintaining an appropriate distance resolution.

[0082] In this embodiment, two microframes with different gate pulse timings are acquired for each pixel, but the number of microframes acquired for each pixel is not limited to two, and at least two microframes may be acquired. Even when three or more microframes with different gate pulse timings are acquired for each pixel, the effect of improving the frame rate can be obtained. For example, when three microframes are acquired for each pixel, the photoelectric conversion device has a third microframe readout unit that acquires the third microframe, and a third microframe adder unit that generates a third subframe by synthesizing the third microframes. The third exposure period is controlled by a third control signal so that the above-mentioned first exposure period, second exposure period, and the third exposure period in which the signal constituting the third microframe is generated are different from each other. The greater the number of microframes with different gate pulse timings that can be acquired for each pixel, the greater the effect of improving the frame rate.

[0083] <Second embodiment> In this embodiment, a different configuration example of the circuit board 21 described in the first embodiment will be described. Descriptions of elements common to the first embodiment may be omitted or simplified as appropriate.

[0084] 11 is a schematic block diagram showing an example of the configuration of the circuit board 21. A plurality of pixel signal processing units 103 are arranged in a plurality of rows and a plurality of columns, and the first memory circuits 211 and the second memory circuits 217 in the pixel signal processing units 103 are connected across all the rows to form a shift register in the column direction. In addition, the plurality of pixel signal processing units 103 are arranged in mirror symmetry when comparing even-numbered columns with odd-numbered columns.

[0085] The first microframe readout unit 323 supplies a clock 1101 to all columns of the shift register to which the first memory circuits 211 are connected across all rows, and reads out data of all columns stored in the first memory circuits 211 in order from the lower row. The second microframe readout unit 327 supplies a clock 1102 to all columns of the shift register to which the second memory circuits 217 are connected across all rows, and reads out data of all columns stored in the second memory circuits 217 in order from the upper row. That is, the readout order of the first microframe readout unit 323 and the readout order of the second microframe readout unit 327 are reversed up and down. In other words, when the first microframe is read out from one side of the pixel array, the second microframe is read out from the opposite side.

[0086] The 1-bit signal constituting the first microframe read by the first microframe read unit 323 is stored in the memory of the first microframe adder unit 324. The first microframe adder unit 324 sequentially adds the value of the microframe to the value stored in the memory each time a microframe is read, thereby generating a first subframe.

[0087] The 1-bit signal constituting the second microframe read by the second microframe read unit 327 is stored in the memory of the second microframe adder unit 328. The second microframe adder unit 328 sequentially adds the value of the microframe to the value stored in the memory each time a microframe is read, thereby generating a second subframe.

[0088] When the second microframe adder 328 stores in memory the microframes read out by the second microframe readout unit 327 in order from the top row, the second microframe adder 328 writes the microframes in the memory in the up-down reverse order to that of the first microframe adder 324. This makes it possible to generate a second subframe whose up-down direction is aligned with that of the first subframe. That is, the data of the row (first row) that is read out first by the second microframe readout unit 327 is written to the last row position in the memory of the second microframe adder 328.

[0089] If the second microframe adder 328 does not write the data into the memory in the up-down reverse order from the first microframe adder 324, the first and second subframes are stored in the up-down reverse order. In this case, when reading the subframes from the memory, the second subframe output unit 329 may read the data from the memory in the up-down reverse order from the first subframe output unit 326. By reading the data in this manner, the second subframe having the same up-down direction as the first subframe can be output to the second subframe group storage unit 333. That is, the data of the row that is output first by the second subframe output unit 329 is read from the position of the last row in the memory in the second microframe adder 328.

[0090] The above describes the case where the second microframe adder 328 writes data to the memory in upside down order and the case where the second subframe output unit 329 reads data from the memory in upside down order. In addition to the above, the first microframe adder 324 may write data to the memory in upside down order, and the first subframe output unit 326 may read data from the memory in upside down order.

[0091] In this embodiment, the first memory circuit 211 and the second memory circuit 217 are configured with shift registers connected across all rows, and the first microframe readout unit 323 and the second microframe readout unit 327 have different readout orders for each row. In this case, too, the first microframe adder 324 to the second microframe adder 328 are written to the memory in a top-to-bottom reversed order, thereby making it possible to output subframes with the same top-to-bottom direction. Alternatively, the first subframe output unit 326 to the second subframe output unit 329 are read from the memory in a top-to-bottom reversed order, thereby making it possible to output subframes with the same top-to-bottom direction. By making the directions of the output subframes the same, it is possible to eliminate the need for top-to-bottom inversion in later processing by the signal processing circuit 33 or an external system (not shown).

[0092] <Third embodiment> As a modification of the second embodiment, an example in which a plurality of pixel signal processing units 103 are arranged in a translationally symmetrical manner when comparing even-numbered columns with odd-numbered columns is shown in Fig. 12. Descriptions of elements common to the second embodiment will be omitted as appropriate.

[0093] In this embodiment, when the read order of each row differs between the microframe readout units, the corresponding microframe adder units can be written to the memory in up-down reversed order to output subframes with the same vertical direction. Alternatively, the corresponding subframe output units can be read from the memory in up-down reversed order to output subframes with the same vertical direction.

[0094] <Fourth embodiment> In this embodiment, another example of the gate pulse timing will be described, which is different from that described in the first embodiment. Descriptions of elements common to the first embodiment may be omitted or simplified as appropriate.

[0095] FIG. 13(a) is a schematic diagram showing an arrangement of a pixel group in a pixel array according to this embodiment, and FIG. 13(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0096] The pixel array of this embodiment includes a first pixel group 34R ("R" in FIG. 13(a)), a second pixel group 34G ("G" in FIG. 13(a)), and a third pixel group 34B ("B" in FIG. 13(a)). The first pixel group 34R, the second pixel group 34G, and the third pixel group 34B are configured to have sensitivity to light of different wavelengths. More specifically, the first pixel group 34R (first photoelectric conversion element) is provided with a first color filter that transmits red light of a wavelength of 700 nm (first wavelength). The second pixel group 34G (second photoelectric conversion element) is provided with a second color filter that transmits green light of a wavelength of 550 nm (second wavelength). The third pixel group 34B (third photoelectric conversion element) is provided with a third color filter that transmits blue light of a wavelength of 430 nm (third wavelength).

[0097] The pulsed light source 311 of this embodiment is configured to emit light of an infrared wavelength. "Light emission (infrared wavelength)" in FIG. 13(b) indicates the timing of light emission of an infrared wavelength by the pulsed light source 311. "P_G1" and "P_G2" indicate the input timing of a plurality of types of gate pulses input from the gate pulse generating unit 322 to the imaging unit 321. "P_G1" corresponds to the control signal input to the first gating circuit 216 via the drive line 215 in FIG. 5. "P_G2" corresponds to the control signal input to the second gating circuit 222 via the drive line 221 in FIG. 5.

[0098] A gate pulse G03 is input to the first gating circuit 216 connected to the first memory circuit 211 at a timing synchronized with the emission timing L02 of the infrared wavelength light. A gate pulse G04 is input to the second gating circuit 222 connected to the second memory circuit 217 for a predetermined exposure period during which imaging is performed.

[0099] As a result, signals including information on each of the colors red, green, and blue are obtained in the second memory circuit 217, and a color image can be generated using these signals. Note that the three colors red, green, and blue are merely examples, and a color image can be generated as long as the acquired signal includes information on a plurality of colors in the visible range. Also, as in the first embodiment, distance information is obtained in the first memory circuit 211 by inputting a gate pulse G03 at a timing synchronized with the light emission.

[0100] In addition, the first color filter that transmits red light with a wavelength of 700 nm (first wavelength), the second color filter that transmits green light with a wavelength of 550 nm (second wavelength), and the third color filter that transmits blue light with a wavelength of 430 nm (third wavelength) are generally known to also transmit light with an infrared wavelength (fourth wavelength). Therefore, the first pixel group 34R, the second pixel group 34G, and the third pixel group 34B are also sensitive to light with an infrared wavelength. Therefore, when the pulse light source 311 emits light with an infrared wavelength, it is possible to obtain a signal for distance measurement. At that time, a general photoelectric conversion device that only performs imaging is equipped with an infrared light cut filter that cuts infrared wavelengths, but in this embodiment, an infrared light cut filter is not installed. In addition, when the wavelength used in distance measurement is an infrared wavelength of 1000 nm, it is more preferable to install a filter that cuts the range from the first wavelength of 700 nm to the infrared wavelength of 1000 nm.

[0101] As described above, according to the configuration of this embodiment, it is possible to obtain a signal for distance measurement and a signal for generating a color image during the same subframe period, thereby providing a photoelectric conversion device that can obtain a color image and a distance image while maintaining the distance resolution and frame rate.

[0102] <Fifth embodiment> In this embodiment, as a modification of the fourth embodiment, an example of a configuration in which a recharge pulse for resetting a photoelectric conversion element can be input to a pixel circuit will be described. Descriptions of elements common to the first to fourth embodiments may be omitted or simplified as appropriate.

[0103] FIG. 14 is a schematic block diagram of a pixel according to this embodiment. In FIG. 14, a transistor 223 is disposed as a specific example of the quench element 202 in FIG. 5. The transistor 223 is an NMOS transistor. The source of the transistor 223 is connected to a connection node between the APD 201 and the input terminal of the waveform shaping unit 210. The drain of the transistor 223 is supplied with a voltage VH. The gate of the transistor 223 is input with a recharge pulse from the first vertical scanning circuit 110 to the second vertical scanning circuit 116 via a drive line 224. As described above, when avalanche multiplication occurs in the APD 201 and an avalanche current flows, the voltage supplied to the APD 201 drops (quench operation). When a high-level recharge pulse is input to the transistor 223, the transistor 223 turns on, the cathode potential of the APD 201 is reset, and a voltage capable of avalanche multiplication is applied to the APD 201. This operation is called a recharge operation. It should be noted that the transistor 223 may be a PMOS transistor, in which case the polarity of the recharge pulse input to the transistor 223 is reversed.

[0104] 15 is a drive timing diagram showing the timing of the gate pulse and the recharge pulse according to this embodiment. The configuration of the pixel group in the pixel array of this embodiment is the same as that of the fourth embodiment. Also, the pulse light source 311 of this embodiment is configured to be able to emit light of an infrared wavelength, similar to the fourth embodiment. "P_R" indicates a recharge pulse.

[0105] The recharge pulse synchronized with the light emission timing L03 is referred to as recharge pulse R01. The recharge pulse R01 goes high after a predetermined time has elapsed since the pulse light source 311 emits light.

[0106] After the recharge pulse R01 falls, the gate pulse G05 input to the first gating circuit 216 connected to the first memory circuit 211 goes to high level. Also, the gate pulse G06 input to the second gating circuit 222 connected to the second memory circuit 217 goes to high level. In the timing diagram shown in Fig. 15, the period during which the gate pulse G06 is at high level is longer than the period during which the gate pulse G05 is at high level.

[0107] In the second memory circuit 217, the period from the rising edge of the recharge pulse R01 to the falling edge of the gate pulse G06 is the photon detection period for capturing a color image. That is, in the configuration of this embodiment, the timing of the recharge pulse R01 and the gate pulse G06 is appropriately set, so that the charge accumulation time in capturing a color image can be controlled. In the configuration of the third embodiment, since the gate pulse G04 for capturing a color image is maintained at a high level, when the subject is a moving object, blurring of the image or mixing of signals from different subjects may occur. However, in the configuration of this embodiment, the influence of these factors can be reduced by appropriately setting the charge accumulation time in capturing a color image. Therefore, according to this embodiment, in addition to obtaining the same effects as the fourth embodiment, the image quality can be further improved.

[0108] Sixth embodiment An example of an electronic device to which the photoelectric conversion device 100 is applied is a digital camera. In digital cameras, an AE (Automatic Exposure) function is known that controls automatic exposure so that a captured image is appropriately bright regardless of the brightness of the subject. In this embodiment, an example of a configuration that can further implement the AE function when capturing a color image in the fifth embodiment will be described. Descriptions of elements common to the first to fifth embodiments may be omitted or simplified as appropriate.

[0109] To realize the AE function, the signal processing circuit 33 acquires brightness information of the subject, such as an integrated value of pixel values, from the light receiving device 32. Based on the acquired brightness information of the subject, the signal processing circuit 33 controls the gate pulse generating unit to adjust the gate pulse width for capturing a color image.

[0110] Fig. 16(a) is a drive timing diagram showing the timing of the gate pulse and the recharge pulse according to this embodiment at low illuminance, and Fig. 16(b) is a drive timing diagram showing the timing of the gate pulse and the recharge pulse according to this embodiment at high illuminance. The configuration of the pixel group in the pixel array of this embodiment is the same as that of the fourth embodiment. Also, the pulse light source 311 of this embodiment is configured to emit light of an infrared wavelength, similar to the fourth embodiment.

[0111] 16(a), a recharge pulse synchronized with the light emission timing L04 is set as recharge pulse R02. Recharge pulse R02 goes high after a predetermined time has elapsed since the pulse light source 311 emits light.

[0112] After the recharge pulse R02 falls, the gate pulse G07 input to the first gating circuit 216 connected to the first memory circuit 211 goes to high level. Also, the gate pulse G08 input to the second gating circuit 222 connected to the second memory circuit 217 goes to high level.

[0113] In the second memory circuit 217, the period from the falling edge of the recharge pulse R02 to the falling edge of the gate pulse G08 is the photon detection period for capturing a color image. At times of low illuminance, the high level period of the gate pulse G08 is lengthened to adjust the charge accumulation time for capturing a color image to be longer, thereby realizing the AE function.

[0114] 16(b), a recharge pulse synchronized with the light emission timing L05 is set as recharge pulse R03. Recharge pulse R03 goes high after a predetermined time has elapsed since the pulse light source 311 emits light.

[0115] After the recharge pulse R03 falls, the gate pulse G09 input to the first gating circuit 216 connected to the first memory circuit 211 goes to high level. Also, the gate pulse G10 input to the second gating circuit 222 connected to the second memory circuit 217 goes to high level.

[0116] In the second memory circuit 217, the period from the falling edge of the recharge pulse R03 to the falling edge of the gate pulse G10 is the photon detection period for capturing a color image. At times of high illuminance, the high level period of the gate pulse G10 is shortened and the charge accumulation time for capturing a color image is adjusted to be shorter, thereby realizing the AE function.

[0117] As described above, the high level period of the gate pulse used for distance measurement can be set to a fixed width, and the high level period of the gate pulse used for imaging can be variably controlled by AE control. Therefore, according to this embodiment, in addition to obtaining the same effects as the fifth embodiment, exposure adjustment is performed automatically, and the captured image becomes an appropriately bright image regardless of the brightness of the subject.

[0118] Seventh embodiment The photoelectric conversion system according to this embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing a schematic configuration of the photoelectric conversion system according to this embodiment.

[0119] The photoelectric conversion devices (imaging devices) described in the first to sixth embodiments are applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites. Camera modules equipped with an optical system such as a lens and an imaging device are also included in photoelectric conversion systems. FIG. 17 illustrates a block diagram of a digital still camera as an example of these.

[0120] 17 includes an image pickup device 1004, which is an example of a photoelectric conversion device, and a lens 1002 that forms an optical image of a subject on the image pickup device 1004. The system further includes an aperture 1003 that varies the amount of light passing through the lens 1002, and a barrier 1001 that protects the lens 1002. The lens 1002 and the aperture 1003 form an optical system that focuses light on the image pickup device 1004. The image pickup device 1004 is a photoelectric conversion device (image pickup device) according to any one of the above embodiments, and converts the optical image formed by the lens 1002 into an electrical signal.

[0121] The photoelectric conversion system also has a signal processing unit 1007 which is an image generating unit that generates an image by processing an output signal output from the imaging device 1004. The signal processing unit 1007 performs various corrections and compression as necessary to output image data. The signal processing unit 1007 may be formed on a semiconductor substrate on which the imaging device 1004 is provided, or may be formed on a semiconductor substrate separate from the imaging device 1004. Moreover, the imaging device 1004 and the signal processing unit 1007 may be formed on the same semiconductor substrate.

[0122] The photoelectric conversion system further includes a memory unit 1010 for temporarily storing image data, and an external interface unit (external I / F unit) 1013 for communicating with an external computer or the like. The photoelectric conversion system further includes a recording medium 1012 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 1011 for recording or reading out data on the recording medium 1012. The recording medium 1012 may be built into the photoelectric conversion system, or may be removable.

[0123] The photoelectric conversion system further includes an overall control / calculation unit 1009 that performs various calculations and controls the entire digital still camera, and a timing generation unit 1008 that outputs various timing signals to the image capture device 1004 and the signal processing unit 1007. Here, the timing signals and the like may be input from outside, and the photoelectric conversion system only needs to include at least the image capture device 1004 and the signal processing unit 1007 that processes the output signal output from the image capture device 1004.

[0124] The imaging device 1004 outputs an imaging signal to a signal processing unit 1007. The signal processing unit 1007 performs predetermined signal processing on the imaging signal output from the imaging device 1004, and outputs image data. The signal processing unit 1007 generates an image using the imaging signal.

[0125] In this way, according to this embodiment, it is possible to realize a photoelectric conversion system to which the photoelectric conversion device (imaging device) according to any one of the above embodiments is applied.

[0126] <Eighth embodiment> The photoelectric conversion system and the moving object of this embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram showing the configuration of the photoelectric conversion system and the moving object of this embodiment.

[0127] FIG. 18(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 1300 has an image capture device 1310. The image capture device 1310 is a photoelectric conversion device (image capture device) described in any of the above embodiments. The photoelectric conversion system 1300 has an image processing unit 1312 that performs image processing on a plurality of image data acquired by the image capture device 1310. The photoelectric conversion system 1300 also has a distance acquisition unit 1316 that calculates a distance to an object, and a collision determination unit 1318 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the distance acquisition unit 1316 may acquire distance information to the object using ToF (Time Of Flight), or may acquire distance information using parallax information or the like. That is, the distance information is information related to parallax, defocus amount, distance to the object, and the like. The collision determination unit 1318 may determine the possibility of collision using any of these distance information. The distance information acquisition means may be realized by dedicated hardware or may be realized by a software module. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or may be realized by a combination of these.

[0128] The photoelectric conversion system 1300 is connected to a vehicle information acquisition device 1320, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. The photoelectric conversion system 1300 is also connected to an ECU 1330, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 1318. The photoelectric conversion system 1300 is also connected to an alarm device 1340 that issues an alarm to the driver based on the judgment result of the collision judgment unit 1318. For example, when the judgment result of the collision judgment unit 1318 indicates that a collision is highly likely, the ECU 1330 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 1340 warns the user by sounding an alarm, displaying alarm information on a screen of a car navigation system, etc., or vibrating a seat belt or steering wheel.

[0129] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are imaged by the photoelectric conversion system 1300. Fig. 18(b) shows the photoelectric conversion system when imaging the area in front of the vehicle (imaging range 1350). A vehicle information acquisition device 1320 sends instructions to the photoelectric conversion system 1300 or the imaging device 1310. This configuration can further improve the accuracy of distance measurement.

[0130] Although an example of control to avoid collision with other vehicles has been described above, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to avoid going out of a lane, and the like. Furthermore, the photoelectric conversion system is not limited to vehicles such as automobiles, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. The moving body includes one or both of a driving force generating unit that generates a driving force mainly used for the movement of the moving body, and a rotating body mainly used for the movement of the moving body. The driving force generating unit can be an engine, a motor, and the like. The rotating body can be a tire, a wheel, a screw of a ship, a propeller of an aircraft, and the like. In addition, the present invention can be applied not only to moving bodies, but also to devices that widely use object recognition, such as an intelligent transport system (ITS).

[0131] <Ninth embodiment> The photoelectric conversion system of this embodiment will be described with reference to Fig. 19. Fig. 19 is a diagram showing an example of a schematic configuration of an endoscopic surgery system which is the photoelectric conversion system of this embodiment.

[0132] 19 shows a state in which an operator (doctor) 1131 is performing surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1150. As shown in the figure, the endoscopic surgery system 1150 is composed of an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.

[0133] The endoscope 1100 is composed of a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. In the illustrated example, the endoscope 1100 is configured as a so-called rigid lens barrel having a rigid lens barrel 1101, but the endoscope 1100 may be configured as a so-called flexible lens barrel having a flexible lens barrel.

[0134] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in a body cavity of a patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0135] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device may be any of the photoelectric conversion devices (imaging devices) described in the above-described embodiments. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.

[0136] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0137] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .

[0138] The light source device 1203 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when photographing an operation site or the like.

[0139] The input device 1137 is an input interface for the endoscopic surgery system 1150. A user can input various information and instructions to the endoscopic surgery system 1150 via the input device 1137.

[0140] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.

[0141] The light source device 1203 that supplies irradiation light to the endoscope 1100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 1203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 1102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0142] Furthermore, the light source device 1203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.

[0143] The light source device 1203 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0144] <Tenth embodiment> The photoelectric conversion system of this embodiment will be described with reference to Figs. 20(a) and (b). Fig. 20(a) describes glasses 1600 (smart glasses) which are the photoelectric conversion system of this embodiment. The glasses 1600 have a photoelectric conversion device 1602. The photoelectric conversion device 1602 is the photoelectric conversion device (imaging device) described in each of the above embodiments. A display device including a light emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. A combination of multiple types of photoelectric conversion devices may also be used. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in Fig. 20(a).

[0145] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for focusing light on the photoelectric conversion device 1602.

[0146] FIG. 20(b) illustrates glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The lens 1611 is formed with an optical system for projecting light emitted from the photoelectric conversion device in the control device 1612 and the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used for detecting the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By having a reduction means for reducing light from the infrared light emission unit to the display unit in a planar view, deterioration of image quality is reduced.

[0147] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.

[0148] More specifically, the gaze detection process is performed based on the pupil-corneal reflex method. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.

[0149] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information about the user's line of sight from the photoelectric conversion device.

[0150] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. In the display area of ​​the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.

[0151] The display area may have a first display area and a second display area different from the first display area, and a high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be received from an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.

[0152] In addition, AI may be used to determine the first field of view area and the area with high priority. The AI ​​may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI ​​program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.

[0153] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.

[0154] <Eleventh embodiment> The above-described photoelectric conversion device and photoelectric conversion system may be applied to electronic devices such as so-called smartphones and tablets.

[0155] 21(a) and 21(b) are diagrams showing an example of an electronic device 1500 equipped with a photoelectric conversion device. Fig. 21(a) shows the front side of the electronic device 1500, and Fig. 21(b) shows the rear side of the electronic device 1500.

[0156] 21(a), a display 1510 for displaying an image is disposed in the center of the surface of the electronic device 1500. Then, along the upper side of the surface of the electronic device 1500, front cameras 1521 and 1522 using photoelectric conversion devices, an IR light source 1530 that emits infrared light, and a visible light source 1540 that emits visible light are disposed.

[0157] Also, as shown in FIG. 21(b), along the upper edge of the back surface of the electronic device 1500, rear cameras 1551 and 1552 using photoelectric conversion devices, an IR light source 1560 that emits infrared light, and a visible light source 1570 that emits visible light are arranged.

[0158] In the electronic device 1500 configured in this manner, by applying the above-mentioned photoelectric conversion device, for example, it is possible to capture a higher quality image. Note that the photoelectric conversion device can also be applied to other electronic devices such as infrared sensors, distance measuring sensors using active infrared light sources, security cameras, and personal or biometric authentication cameras. This can improve the accuracy, performance, and the like of these electronic devices.

[0159] As used herein, expressions such as "A or B," "at least one of A and B," "at least one of A or / and B," "one or more of A or / and B," and the like, can include all possible combinations of the listed items unless expressly defined otherwise. That is, the above expressions are understood to disclose all cases including at least one A, at least one B, and both at least one A and at least one B. This applies equally to combinations of three or more elements.

[0160] The above-described embodiments can be modified as appropriate without departing from the technical concept. The disclosure of this specification includes not only what is described in this specification, but also all matters that can be understood from this specification and the drawings attached hereto. The disclosure of this specification also includes the complement of the concepts described in this specification. In other words, if this specification contains a statement that "A is greater than B," for example, this specification can be said to disclose that "A is not greater than B" even if the statement that "A is not greater than B" is omitted. This is because when it is stated that "A is greater than B," it is assumed that the case in which "A is not greater than B" is taken into consideration.

[0161] The disclosure of this embodiment includes the following configurations and methods.

[0162] (Configuration 1) a pixel array in which a plurality of pixels each having a photoelectric conversion element are arranged in an array; a first microframe readout unit that acquires a first microframe based on light incident on each of the plurality of photoelectric conversion elements; a second microframe readout unit that acquires a second microframe based on light incident on each of the plurality of photoelectric conversion elements; a first microframe adder that generates a first subframe by combining the first microframes; a second microframe adder that generates a second subframe by combining the second microframes; a first exposure period during which a signal constituting the first microframe is generated is controlled by a first control signal; a second exposure period during which the signals constituting the second microframe are generated is controlled by a second control signal; A photoelectric conversion device, wherein the first exposure period and the second exposure period are different from each other during an acquisition period of one microframe.

[0163] (Configuration 2) the photoelectric conversion element includes an avalanche photodiode, 2. The photoelectric conversion device according to configuration 1, wherein the signal indicates whether or not a photon is incident on the avalanche photodiode within a period during which the microframe is acquired.

[0164] (Configuration 3) an input terminal of a first gating circuit to which a signal constituting the first microframe is input, and an input terminal of a second gating circuit to which a signal constituting the second microframe is input are connected in parallel to an output terminal of the avalanche photodiode; the first gating circuit is connected to a first counter circuit that counts signals that constitute the first microframe; 3. The photoelectric conversion device according to configuration 2, wherein the second gating circuit is connected to a second counter circuit that counts the signals that constitute the second microframe.

[0165] (Configuration 4) a pixel array in which a plurality of pixels, each having an avalanche photodiode, are arranged in an array; an input terminal of a first gating circuit to which a signal constituting a first microframe is input, the input terminal being connected in parallel to an output terminal of the avalanche photodiode; and an input terminal of a second gating circuit to which a signal constituting a second microframe is input. a first microframe reading unit that is connected to an output terminal of the first gating circuit via a first counter circuit that counts signals that constitute the first microframe and that acquires the first microframe; a second microframe reading unit that is connected to an output terminal of the second gating circuit via a second counter circuit that counts signals that constitute the second microframe and that acquires the second microframe; a first microframe adder that generates a first subframe by combining the first microframes; a second microframe adder that generates a second subframe by combining the second microframes.

[0166] (Configuration 5) 5. The photoelectric conversion device according to any one of configurations 1 to 4, wherein the first subframe is a multi-bit signal.

[0167] (Configuration 6) 6. The photoelectric conversion device according to configuration 5, wherein the first microframe is a 1-bit signal.

[0168] (Configuration 7) The photoelectric conversion device according to any one of configurations 1 to 6, further comprising a first readout unit that reads out the first microframe from any one side of the pixel array.

[0169] (Configuration 8) 8. The photoelectric conversion device according to configuration 7, further comprising a second readout section that reads out the second microframe from an opposite side to the side.

[0170] (Configuration 9) the first microframe readout unit reads out signals based on the incident light in order from pixels arranged in a first row to a last row of the pixel array; A photoelectric conversion device described in any one of configurations 1 to 8, characterized in that the second microframe reading unit reads out signals based on the incident light in the reverse order to the reading by the first microframe reading unit, starting from the pixels arranged in the first row to the last row of the pixel array.

[0171] (Configuration 10) The photoelectric conversion device described in any one of configurations 1 to 9, characterized in that the first microframe addition unit generates a subframe that is upside down when the first microframe reading unit reads out signals based on the incident light in sequence starting from the pixels arranged in the last row of the pixel array.

[0172] (Configuration 11) The photoelectric conversion device described in any one of configurations 1 to 10, characterized in that the second microframe addition unit generates a subframe that is upside down when the second microframe reading unit reads out signals based on the incident light in sequence starting from the pixels arranged in the last row of the pixel array.

[0173] (Configuration 12) 12. The photoelectric conversion device according to any one of configurations 1 to 11, wherein the first microframe is used for distance measurement, and the second microframe is used for imaging.

[0174] (Configuration 13) 13. The photoelectric conversion device according to configuration 12, wherein the first exposure period is constant, and the second exposure period is variably controlled by automatic exposure control.

[0175] (Configuration 14) the pixel array includes a first photoelectric conversion element having sensitivity to a first wavelength, a second photoelectric conversion element having sensitivity to a second wavelength, and a third photoelectric conversion element having sensitivity to a third wavelength; 14. The photoelectric conversion device according to any one of configurations 1 to 13, wherein the first wavelength, the second wavelength, and the third wavelength are wavelengths in the visible region.

[0176] (Configuration 15) 15. The photoelectric conversion device according to any one of configurations 1 to 14, wherein the plurality of photoelectric conversion elements include a photoelectric conversion element sensitive to a fourth wavelength that is an infrared wavelength.

[0177] (Configuration 16) The first exposure period starts when a first time has elapsed since a light source emits light onto an object; The photoelectric conversion device according to any one of configurations 1 to 15, wherein the second exposure period starts at a time when a second time different from the first time has elapsed from the light emission timing.

[0178] (Configuration 17) The photoelectric conversion device according to configuration 5 or 6, wherein the first microframe addition unit generates the multi-bit signal by adding up the value of the signal each time the first microframe is acquired.

[0179] (Configuration 18) a first microframe reading unit that acquires a third microframe based on light incident on each of the plurality of photoelectric conversion elements; a second microframe adder that generates a third subframe by combining the third microframes; a third exposure period during which the signals constituting the third microframe are generated is controlled by a third control signal; A photoelectric conversion device according to any one of configurations 1 to 17, characterized in that, during an acquisition period of one microframe, the first exposure period, the second exposure period, and the third exposure period are different from each other.

[0180] (Configuration 19) A photoelectric conversion device according to any one of configurations 1 to 18, a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

[0181] (Configuration 20) A moving object including the photoelectric conversion device according to any one of configurations 1 to 18, A moving body comprising: a control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device. [Explanation of symbols]

[0182] 211 First memory circuit 217 Second memory circuit 323 First microframe readout unit 324 First microframe adder 327 Second microframe readout unit 328 Second microframe adder

Claims

1. a pixel array in which a plurality of pixels each having a photoelectric conversion element are arranged in an array; a first microframe readout unit that acquires a first microframe based on light incident on each of the plurality of photoelectric conversion elements; a second microframe readout unit that acquires a second microframe based on incident light to each of the plurality of photoelectric conversion elements; a first microframe adder that generates a first subframe by combining the first microframes; a second microframe adder that generates a second subframe by combining the second microframes; a first exposure period during which a signal constituting the first microframe is generated is controlled by a first control signal; a second exposure period during which the signals constituting the second microframe are generated is controlled by a second control signal; A photoelectric conversion device, characterized in that, in an acquisition period of one microframe, the first exposure period and the second exposure period are different from each other.

2. 2. The photoelectric conversion device according to claim 1, wherein the first subframe is a multi-bit signal.

3. 3. The photoelectric conversion device according to claim 2, wherein the first microframe is a 1-bit signal.

4. 2 . The photoelectric conversion device according to claim 1 , further comprising a first readout unit that reads out the first microframe from any one of the sides of the pixel array.

5. 5. The photoelectric conversion device according to claim 4, further comprising a second readout section for reading out the second microframe from an opposite side to the side.

6. the first microframe readout unit reads out signals based on the incident light in order from pixels arranged in a first row to a last row of the pixel array; The photoelectric conversion device according to claim 5, characterized in that the second microframe reading unit reads out signals based on the incident light in the reverse order to the reading by the first microframe reading unit, starting from the pixels arranged in the first row through the last row of the pixel array.

7. The photoelectric conversion device according to claim 4, characterized in that the first microframe adder generates a subframe that is upside down when the first microframe readout unit reads out signals based on the incident light in sequence starting from the pixels arranged in the last row of the pixel array.

8. The photoelectric conversion device according to claim 5, characterized in that the second microframe adder generates a subframe that is upside down when the second microframe readout unit reads out signals based on the incident light in sequence starting from the pixels arranged in the last row of the pixel array.

9. 2. The photoelectric conversion device according to claim 1, wherein the first microframe is used for distance measurement, and the second microframe is used for imaging.

10. 10. The photoelectric conversion device according to claim 9, wherein the first exposure period is constant, and the second exposure period is variably controlled by automatic exposure control.

11. the pixel array includes a first photoelectric conversion element having sensitivity to a first wavelength, a second photoelectric conversion element having sensitivity to a second wavelength, and a third photoelectric conversion element having sensitivity to a third wavelength; 11. The photoelectric conversion device according to claim 9, wherein the first wavelength, the second wavelength, and the third wavelength are wavelengths in a visible region.

12. 2. The photoelectric conversion device according to claim 1, wherein the plurality of photoelectric conversion elements includes a photoelectric conversion element having sensitivity to a fourth wavelength that is an infrared wavelength.

13. the first exposure period starts when a first time has elapsed since a light emission timing of a light source that emits light to an object; 2 . The photoelectric conversion device according to claim 1 , wherein the second exposure period starts when a second time different from the first time has elapsed from the light emission timing.

14. the photoelectric conversion element includes an avalanche photodiode, 2 . The photoelectric conversion device according to claim 1 , wherein the signal indicates whether or not a photon is incident on the avalanche photodiode within a period during which the microframe is acquired.

15. an input terminal of a first gating circuit to which a signal constituting the first microframe is input, and an input terminal of a second gating circuit to which a signal constituting the second microframe is input are connected in parallel to an output terminal of the avalanche photodiode; the first gating circuit is connected to a first counter circuit that counts signals that constitute the first microframe; 15. The photoelectric conversion device according to claim 14, wherein the second gating circuit is connected to a second counter circuit that counts the signals that constitute the second microframe.

16. 3 . The photoelectric conversion device according to claim 2 , wherein the first microframe adder generates the multi-bit signal by adding up the signal value each time the first microframe is acquired.

17. a first microframe readout unit that acquires a third microframe based on incident light to each of the plurality of photoelectric conversion elements; a second microframe adder that generates a third subframe by combining the third microframes; a third exposure period during which the signals constituting the third microframe are generated is controlled by a third control signal; 2 . The photoelectric conversion device according to claim 1 , wherein in an acquisition period of one microframe, the first exposure period, the second exposure period, and the third exposure period are different from one another.

18. a pixel array in which a plurality of pixels, each having an avalanche photodiode, are arranged in an array; an input terminal of a first gating circuit to which a signal constituting a first microframe is input, the input terminal being connected in parallel to an output terminal of the avalanche photodiode; and an input terminal of a second gating circuit to which a signal constituting a second microframe is input. a first microframe reading unit that is connected to an output terminal of the first gating circuit via a first counter circuit that counts signals that constitute the first microframe and that acquires the first microframe; a second microframe reading unit that is connected to an output terminal of the second gating circuit via a second counter circuit that counts signals that constitute the second microframe and that acquires the second microframe; a first microframe adder that generates a first subframe by combining the first microframes; a second microframe adder that generates a second subframe by combining the second microframes.

19. The photoelectric conversion device according to any one of claims 1 to 18, a signal processing unit that generates an image using a signal output from the photoelectric conversion device.

20. A moving object comprising the photoelectric conversion device according to any one of claims 1 to 18, A moving body comprising: a control unit that controls the movement of the moving body using a signal output from the photoelectric conversion device.

Images