Photoelectric conversion device

Abstract

To provide a photoelectric conversion device that improves a frame rate while ensuring appropriate distance resolution.SOLUTION: A photoelectric conversion device has: a plurality of photoelectric conversion elements; an acquisition unit that acquires a micro-frame comprising one-bit signals on the basis of incident light on each of the plurality of photoelectric conversion elements; and a composition unit that combines the plurality of micro-frames to create a sub-frame comprising multi-bit signals. The plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element. In a period for acquiring the one micro-frame, a first exposure time during which creation of the one-bit signals based on the incident light on the first photoelectric conversion element, and a second exposure time during which creation of the one-bit signals based on the incident light on the second photoelectric conversion element, are different from each other.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a photoelectric conversion device. [Background technology]

[0002] Patent Document 1 discloses a distance measuring device that measures the distance to an object by emitting light from a light source and receiving light including reflected light from the object with a light receiving element. In the distance measuring device of Patent Document 1, a SPAD (Single Photon Avalanche Diode) element that multiplies electrons generated by photoelectric conversion to obtain a signal is used as the light receiving element. Patent Document 1 discloses a method of repeatedly performing measurements while changing a gating period during which photons are detected in the SPAD element. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2017 / 0052065 Summary of the Invention [Problem to be solved by the invention]

[0004] In the ranging method disclosed in Patent Document 1, there is a trade-off between distance resolution and frame rate. However, in order to improve ranging performance, there are cases where it is required to ensure both appropriate distance resolution and an improved frame rate.

[0005] An object of the present invention is to provide a photoelectric conversion device that ensures an appropriate distance resolution while improving the frame rate. [Means for solving the problem]

[0006] According to one disclosure of the present specification, there is provided a photoelectric conversion device comprising: a plurality of photoelectric conversion elements; an acquisition unit that acquires a microframe composed of a 1-bit signal based on incident light to each of the plurality of photoelectric conversion elements; and a synthesis unit that generates a subframe composed of a multi-bit signal by synthesizing a plurality of the microframes, wherein the plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element; and during the acquisition period of one of the microframes, a first exposure period in which the 1-bit signal is generated based on the incident light to the first photoelectric conversion element and a second exposure period in which the 1-bit signal is generated based on the incident light to the second photoelectric conversion element are different from each other. Effect of the Invention

[0007] According to the present invention, a photoelectric conversion device is provided that ensures an appropriate distance resolution while improving the frame rate. [Brief description of the drawings]

[0008] [Figure 1] 1 is a hardware block diagram showing an example of a schematic configuration of a distance measuring device according to a first embodiment. [Diagram 2] 1 is a schematic diagram showing the overall configuration of a photoelectric conversion device according to a first embodiment. [Diagram 3] 1 is a schematic block diagram showing an example of the configuration of a sensor substrate according to a first embodiment. [Figure 4] 1 is a schematic block diagram showing an example of the configuration of a circuit board according to a first embodiment. [Diagram 5] 2 is a schematic block diagram showing an example of the configuration of one pixel of a photoelectric conversion unit and a pixel signal processing unit according to the first embodiment. FIG. [Figure 6] 3A to 3C are diagrams illustrating the operation of the avalanche photodiode according to the first embodiment. [Figure 7] 1 is a functional block diagram showing a schematic configuration example of a distance measuring device according to a first embodiment. [Figure 8] 2A to 2C are schematic diagrams for explaining ranging frames, subframes, and microframes according to the first embodiment. [Figure 9] 5 is a flowchart showing the operation of the distance image generating device according to the first embodiment in one ranging frame period. [Figure 10] 1A and 1B are a schematic diagram and a drive timing chart of a pixel array according to the first embodiment. [Figure 11] 11A and 11B are schematic diagrams and drive timing charts of a pixel array according to a second embodiment. [Figure 12] 13A and 13B are a schematic diagram and a drive timing chart of a pixel array according to a third embodiment. [Figure 13] FIG. 13 is a schematic block diagram of a pixel according to a fourth embodiment. [Figure 14] FIG. 13 is a drive timing chart according to the fourth embodiment. [Figure 15] FIG. 13 is a functional block diagram showing a schematic configuration example of a distance measuring device according to a fifth embodiment. [Figure 16] FIG. 13 is a schematic diagram showing an example of an external light map according to the fifth embodiment. [Figure 17] FIG. 13 is a schematic diagram of a device according to a sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same elements or corresponding elements in multiple drawings are denoted by the same reference numerals, and the description thereof may be omitted or simplified.

[0010] [First embodiment] 1 is a hardware block diagram showing an example of the schematic configuration of a distance image generating device 30 according to this embodiment. The distance image generating device 30 has a light emitting device 31, a light receiving device 32, and a signal processing circuit 33. Note that the configuration of the distance image generating device 30 shown in this embodiment is just an example, and is not limited to the configuration shown in the figure.

[0011] The distance image generating device 30 is a device that measures the distance to the object X to be measured using technology such as LiDAR (Light Detection And Ranging). The distance image generating device 30 measures the distance from the distance image generating device 30 to the object X based on the time difference between when light emitted from a light emitting device 31 is reflected by the object X and when it is received by a light receiving device 32. The distance image generating device 30 can also measure the distance to a plurality of points two-dimensionally by emitting laser light to a predetermined distance measurement range including the object X and receiving the reflected light with a pixel array. This allows the distance image generating device 30 to generate and output a distance image.

[0012] The light received by the light receiving device 32 includes ambient light such as sunlight in addition to the reflected light from the object X. Therefore, the distance image generating device 30 performs distance measurement with reduced influence of ambient light by measuring the light incident during each of a plurality of periods (bin periods) and determining that the reflected light is incident during the period when the amount of light is at its peak.

[0013] The light emitting device 31 is a device that emits light such as laser light to the outside of the distance image generating device 30. The signal processing circuit 33 may include a processor that performs arithmetic processing of digital signals, a memory that stores digital signals, etc. The memory may be, for example, a semiconductor memory.

[0014] The light receiving device 32 generates a pulse signal including a pulse based on the incident light. The light receiving device 32 is, for example, a photoelectric conversion device including an avalanche photodiode as a photoelectric conversion element. In this case, when one photon is incident on the avalanche photodiode and an electric charge is generated, one pulse is generated by avalanche multiplication. However, the light receiving device 32 may be, for example, a device using a photoelectric conversion element using another photodiode.

[0015] In this embodiment, the light receiving device 32 includes a pixel array in which a plurality of photoelectric conversion elements (pixels) are arranged to form a plurality of rows and a plurality of columns. Here, a photoelectric conversion device, which is a specific configuration example of the light receiving device 32, will be described with reference to Figs. 2 to 6. The configuration example of the photoelectric conversion device described below is one example. The photoelectric conversion device applicable to the light receiving device 32 is not limited to this, and may be any device capable of realizing the function of Fig. 7 described later.

[0016] FIG. 2 is a schematic diagram showing the overall configuration of a photoelectric conversion device 100 according to this embodiment. The photoelectric conversion device 100 has a sensor substrate 11 (first substrate) and a circuit substrate 21 (second substrate) that are stacked on each other. The sensor substrate 11 and the circuit substrate 21 are electrically connected to each other. The sensor substrate 11 has a pixel region 12 in which a plurality of pixels 101 are arranged to form a plurality of rows and a plurality of columns. The circuit substrate 21 has a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns, and a second circuit region 23 arranged on the periphery of the first circuit region 22. The second circuit region 23 may include a circuit for controlling the plurality of pixel signal processing units 103. The sensor substrate 11 has a light incident surface that receives incident light and a connection surface that faces the light incident surface. The sensor substrate 11 is connected to the circuit substrate 21 on the connection surface side. That is, the photoelectric conversion device 100 is a so-called back-illuminated type.

[0017] In this specification, "planar view" refers to a view from a direction perpendicular to the surface opposite to the light incident surface. Also, a cross section refers to a surface in a direction perpendicular to the surface opposite to the light incident surface of the sensor substrate 11. Note that the light incident surface may be rough when viewed microscopically, and in such a case, the planar view is defined based on the light incident surface when viewed macroscopically.

[0018] In the following description, the sensor substrate 11 and the circuit substrate 21 are described as being diced chips, but the sensor substrate 11 and the circuit substrate 21 are not limited to being chips. For example, the sensor substrate 11 and the circuit substrate 21 may be wafers. In addition, when the sensor substrate 11 and the circuit substrate 21 are diced chips, the photoelectric conversion device 100 may be manufactured by stacking them in a wafer state and then dicing them, or may be manufactured by stacking them after dicing.

[0019] 3 is a schematic block diagram showing an example of the arrangement of the sensor substrate 11. A plurality of pixels 101 arranged in a plurality of rows and a plurality of columns are arranged in the pixel region 12. Each of the plurality of pixels 101 has a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter referred to as APD) as a photoelectric conversion element within the substrate.

[0020] The conductivity type of the charge pair generated in the APD and used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type in which the charge of the same polarity as the signal charge is the majority carrier. The conductivity type opposite to the first conductivity type, that is, the conductivity type in which the charge of the opposite polarity to the signal charge is the majority carrier, is called the second conductivity type. In the APD described below, the anode of the APD is at a fixed potential, and a signal is taken out from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is an N-type semiconductor region, and the semiconductor region of the second conductivity type is a P-type semiconductor region. The cathode of the APD may be at a fixed potential, and the signal may be taken out from the anode of the APD. In this case, the semiconductor region of the first conductivity type is a P-type semiconductor region, and the semiconductor region of the second conductivity type is an N-type semiconductor region. In the following, a case in which one node of the APD is at a fixed potential will be described, but the potentials of both nodes may be fluctuating.

[0021] 4 is a schematic block diagram showing a configuration example of the circuit board 21. The circuit board 21 has a first circuit area 22 in which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns.

[0022] Further, a vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, pixel output signal lines 113, an output circuit 114, and a control signal generating unit 115 are arranged on the circuit board 21. The multiple photoelectric conversion units 102 shown in Fig. 3 and the multiple pixel signal processing units 103 shown in Fig. 4 are electrically connected to each other via connection wiring provided for each pixel 101.

[0023] The control signal generating unit 115 is a control circuit that generates control signals for driving the vertical scanning circuit 110, the horizontal scanning circuit 111, and the readout circuit 112, and supplies these circuits with these signals. In this way, the control signal generating unit 115 controls the drive timing of each circuit.

[0024] The vertical scanning circuit 110 supplies a control signal to each of the pixel signal processing units 103 based on the control signal supplied from the control signal generation unit 115. The vertical scanning circuit 110 supplies a control signal to each pixel signal processing unit 103 for each row via a drive line provided for each row of the first circuit area 22. As will be described later, there may be multiple drive lines for each row. The vertical scanning circuit 110 may include logic circuits such as a shift register and an address decoder. In this way, the vertical scanning circuit 110 selects a row for outputting a signal from the pixel signal processing unit 103.

[0025] The signal output from the photoelectric conversion unit 102 of the pixel 101 is processed by the pixel signal processing unit 103. The pixel signal processing unit 103 counts the number of pulses output from the APD included in the photoelectric conversion unit 102 to obtain and hold a digital signal having multiple bits.

[0026] It is not necessary that one pixel signal processing unit 103 is provided for each pixel 101. For example, one pixel signal processing unit 103 may be shared by a plurality of pixels 101. In this case, the pixel signal processing unit 103 provides a signal processing function to each pixel 101 by sequentially processing the signals output from each photoelectric conversion unit 102.

[0027] The horizontal scanning circuit 111 supplies a control signal to the readout circuit 112 based on the control signal supplied from the control signal generation unit 115. The pixel signal processing unit 103 is connected to the readout circuit 112 via a pixel output signal line 113 provided for each column of the first circuit region 22. The pixel output signal line 113 of one column is shared by a plurality of pixel signal processing units 103 of the corresponding column. The pixel output signal line 113 includes a plurality of wirings, and has at least a function of outputting a digital signal from each pixel signal processing unit 103 to the readout circuit 112 and a function of supplying a control signal for selecting a column for outputting a signal to the pixel signal processing unit 103. The readout circuit 112 outputs a signal to a storage unit or a signal processing unit outside the photoelectric conversion device 100 via the output circuit 114 based on the control signal supplied from the control signal generation unit 115.

[0028] The photoelectric conversion units 102 in the pixel region 12 may be arranged one-dimensionally. Moreover, the function of the pixel signal processing unit 103 does not necessarily have to be provided for each pixel 101. For example, one pixel signal processing unit 103 may be shared by a plurality of pixels 101. In this case, the pixel signal processing unit 103 provides a signal processing function to each pixel 101 by sequentially processing signals output from each photoelectric conversion unit 102.

[0029] 3 and 4, a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged is arranged in a region overlapping the pixel region 12 in a plan view. A vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generating unit 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. In the circuit substrate 21, a second circuit region 23 (described above in FIG. 2) in which the vertical scanning circuit 110, the horizontal scanning circuit 111, the readout circuit 112, the output circuit 114, and the control signal generating unit 115 are arranged is arranged in a region overlapping the non-pixel region in a plan view.

[0030] The arrangement of the pixel output signal lines 113, the readout circuits 112, and the output circuits 114 are not limited to those shown in Fig. 3. For example, the pixel output signal lines 113 may be arranged to extend in the row direction and shared by a plurality of pixel signal processing units 103 in the corresponding row. The readout circuits 112 may be arranged so that the pixel output signal lines 113 in each row are connected to each other.

[0031] Fig. 5 is a schematic block diagram showing a configuration example of one pixel of the photoelectric conversion unit 102 and pixel signal processing unit 103 according to this embodiment. Fig. 5 shows a more specific configuration example including a connection relationship between the photoelectric conversion unit 102 arranged on the sensor substrate 11 and the pixel signal processing unit 103 arranged on the circuit substrate 21. Note that in Fig. 5, the drive lines between the vertical scanning circuit 110 and the pixel signal processing unit 103 in Fig. 4 are shown as drive lines 213, 214, and 215.

[0032] The photoelectric conversion unit 102 has an APD 201. The pixel signal processing unit 103 has a quenching element 202, a waveform shaping unit 210, a counter circuit 211, a selection circuit 212, and a gating circuit 216. It is sufficient that the pixel signal processing unit 103 has at least one of the waveform shaping unit 210, the counter circuit 211, the selection circuit 212, and the gating circuit 216.

[0033] The APD 201 generates charges according to incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 201. A cathode of the APD 201 is connected to a first terminal of the quench element 202 and an input terminal of the waveform shaping unit 210. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 201. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD 201 such that the APD 201 performs avalanche multiplication. When charges are generated by incident light in the APD 201 to which the reverse bias voltage is supplied, the charges undergo avalanche multiplication, generating an avalanche current.

[0034] In addition, there are two operation modes when a reverse bias voltage is supplied to the APD 201: Geiger mode and linear mode. The Geiger mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is greater than the breakdown voltage, and the linear mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is close to or less than the breakdown voltage.

[0035] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the voltage VL (first voltage) is −30 V, and the voltage VH (second voltage) is 1 V. The APD 201 may be operated in either linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger than that of an APD in linear mode, and the effect of avalanche multiplication is more pronounced, so a SPAD is preferable.

[0036] The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication. The quench element 202 suppresses the voltage supplied to the APD 201 to suppress avalanche multiplication (quench operation). The quench element 202 also returns the voltage supplied to the APD 201 to the voltage VH by passing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 202 can be, for example, a resistive element.

[0037] The waveform shaping unit 210 shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used as the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 5, the waveform shaping unit 210 may be a circuit in which a plurality of inverters are connected in series, or may be another circuit having a waveform shaping effect.

[0038] The gating circuit 216 is a circuit that performs gating such that the pulse signal output from the waveform shaping unit 210 passes only for a predetermined period. During the period during which the pulse signal can pass through the gating circuit 216, photons incident on the APD 201 are counted by the counter circuit 211 at the rear stage. Therefore, the gating circuit 216 controls the exposure period during which a signal is generated based on the incident light in the pixel 101. The period during which the pulse signal passes is controlled by a control signal supplied from the vertical scanning circuit 110 via the drive line 215. FIG. 5 shows an example in which one AND circuit is used as the gating circuit 216. A pulse signal and a control signal are input to two input terminals of the AND circuit. The AND circuit outputs the logical product of these to the counter circuit 211. Note that the gating circuit 216 may have a circuit configuration other than an AND circuit as long as it can realize gating. In addition, the waveform shaping unit 210 and the gating circuit 216 may be integrated by using a logic circuit such as a NAND circuit.

[0039] The counter circuit 211 counts the pulse signal output from the waveform shaping unit 210 via the gating circuit 216, and holds a digital signal indicating the count value. When a control signal is supplied from the vertical scanning circuit 110 via the drive line 213, the counter circuit 211 resets the signal it holds.

[0040] A control signal is supplied to the selection circuit 212 from the vertical scanning circuit 110 shown in Fig. 4 via a drive line 214 shown in Fig. 5. In response to this control signal, the selection circuit 212 switches between electrical connection and non-connection between the counter circuit 211 and the pixel output signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to a value held in the counter circuit 211.

[0041] 5, the selection circuit 212 switches between electrical connection and disconnection between the counter circuit 211 and the pixel output signal line 113, but the method of controlling the signal output to the pixel output signal line 113 is not limited to this. For example, a switch such as a transistor may be disposed at a node between the quench element 202 and the APD 201, between the photoelectric conversion unit 102 and the pixel signal processing unit 103, or the like, and the signal output to the pixel output signal line 113 may be controlled by switching between electrical connection and disconnection. Alternatively, the signal output to the pixel output signal line 113 may be controlled by changing the value of the voltage VH or voltage VL supplied to the photoelectric conversion unit 102 using a switch such as a transistor.

[0042] Figures 6(a), 6(b) and 6(c) are diagrams for explaining the operation of the APD 201 according to this embodiment. Figure 6(a) is a diagram showing the APD 201, the quench element 202 and the waveform shaping unit 210 extracted from Figure 5. As shown in Figure 6(a), the connection node of the APD 201, the quench element 202 and the input terminals of the waveform shaping unit 210 is referred to as nodeA. Also, as shown in Figure 6(a), the output side of the waveform shaping unit 210 is referred to as nodeB.

[0043] FIG. 6(b) is a graph showing the time change of the potential of nodeA in FIG. 6(a). FIG. 6(c) is a graph showing the time change of the potential of nodeB in FIG. 6(a). In the period from time t0 to time t1, a voltage of VH-VL is applied to the APD 201 in FIG. 6(a). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201. As a result, an avalanche current flows through the quench element 202, and the potential of nodeA drops. After that, the amount of potential drop further increases, and the voltage applied to the APD 201 gradually decreases. Then, at time t2, the avalanche multiplication in the APD 201 stops. As a result, the voltage level of nodeA does not drop below a certain value. After that, in the period from time t2 to time t3, a current that compensates for the voltage drop flows from the node of voltage VH to nodeA, and at time t3, nodeA settles to its original potential.

[0044] In the above process, the potential of node B becomes high during the period when the potential of node A is lower than a certain threshold. In this way, the waveform of the drop in the potential of node A caused by the incidence of a photon is shaped by the waveform shaping unit 210 and output as a pulse to node B.

[0045] Next, the overall configuration and operation of the distance image generating device 30 will be described in more detail. Fig. 7 is a functional block diagram showing an example of a schematic configuration of the distance image generating device 30 according to this embodiment. Fig. 7 shows a more detailed configuration of the light emitting device 31, the light receiving device 32, and the signal processing circuit 33 described in Fig. 1.

[0046] The light emitting device 31 has a pulsed light source 311 and a light source control unit 312. The pulsed light source 311 is a light source such as a semiconductor laser device that emits pulsed light throughout the entire distance measurement range. The pulsed light source 311 may be a surface light source such as a surface emitting laser. The light source control unit 312 is a control circuit that controls the light emission timing of the pulsed light source 311.

[0047] The light receiving device 32 has an imaging section 321, a gate pulse generating section 322, a microframe readout section 323, a microframe adder 324, an addition count control section 325, and a subframe output section 326. As described above, the imaging section 321 may be a photoelectric conversion device including a pixel array in which pixel circuits including the APD 201 are arranged two-dimensionally. This enables the distance image generating device 30 to obtain a two-dimensional distance image.

[0048] The gate pulse generating unit 322 is a control circuit that outputs a control signal that controls the drive timing of the imaging unit 321. The gate pulse generating unit 322 transmits and receives a control signal to and from the light source control unit 312 to synchronously control the pulse light source 311 and the imaging unit 321. This enables imaging in which the time difference between the time when light is emitted from the pulse light source 311 and the time when the light is received by the imaging unit 321 is controlled. In this embodiment, the gate pulse generating unit 322 drives the imaging unit 321 with a global gate. The global gate drive is a drive method in which incident light is detected simultaneously in several pixels (pixel groups) in the imaging unit 321 during the same exposure period, based on the emission time of the pulse light from the pulse light source 311. In the global gate drive of this embodiment, incident light is repeatedly detected while the collective exposure timing is shifted sequentially. As a result, each pixel of the imaging unit 321 simultaneously generates a 1-bit signal indicating the presence or absence of incident photons in each of a plurality of exposure periods.

[0049] This global gate driving is realized by inputting a high-level signal during a gating period to the input terminals of the gating circuits 216 of the multiple pixels 101 based on a control signal from the gate pulse generating unit 322. Note that in the process described below, one group of the multiple pixels 101 detects incident light during the same exposure period, and another group of the multiple pixels 101 detects incident light during a different exposure period, and such a driving method is also considered to be included in the global gate driving.

[0050] The microframe readout unit 323, the microframe adder 324, and the addition count control unit 325 are signal processing circuits that read out 1-bit signals constituting a microframe from the imaging unit 321 and perform predetermined signal processing. Details of the operation of each of these units will be described later with reference to FIG. 9. The subframe output unit 326 is an interface that outputs a signal in a predetermined standard from the light receiving device 32 to the signal processing circuit 33. The subframe output unit 326 transmits a signal from the memory in the light receiving device 32 to the memory in the signal processing circuit 33. The functions of each of these units can be realized by the counter circuit 211, the selection circuit 212, the gating circuit 216 in FIG. 5, and the readout circuit 112 and the output circuit 114 in FIG. 4, etc.

[0051] Signal processing circuit 33 has a subframe group storage unit 331 and a distance image generation unit 332. Signal processing circuit 33 is a computer including a processor that operates as distance image generation unit 332, and a memory that operates as subframe group storage unit 331. The operation of each of these units will also be described later with reference to FIG.

[0052] Next, prior to describing the driving method of this embodiment, the configurations of ranging frames, subframes, and microframes will be described with reference to Fig. 8. Fig. 8 shows a schematic diagram of acquisition periods of ranging frames corresponding to distance images, subframes used to generate ranging frames, and microframes used to generate subframes, arranged in blocks in the horizontal direction. The horizontal direction in Fig. 8 indicates the passage of time, and one block indicates the acquisition period of one ranging frame, subframe, or microframe.

[0053] The ranging frame F1 corresponds to one distance image. That is, the ranging frame F1 has information corresponding to the distance to the object X, calculated from the time difference between when light is emitted and when it is received, for each of a plurality of pixels. In this embodiment, it is assumed that the distance image is acquired as a video, and acquisition of one ranging frame F1 is repeated every time one ranging frame period T1 elapses.

[0054] One ranging frame F1 is generated from multiple subframes F2. One ranging frame period T1 includes multiple subframe periods T2. One subframe F2 is repeatedly acquired every time one subframe period T2 elapses. The subframe F2 is composed of a multi-bit signal that corresponds to the amount of light incident in the subframe period T2.

[0055] One subframe F2 is generated from multiple microframes F3. One subframe period T2 includes multiple microframe periods T3. One microframe F3 is repeatedly acquired every time one microframe period T3 elapses. The microframe F3 is composed of a 1-bit signal indicating the presence or absence of incident light to the photoelectric conversion element in the microframe period T3. One subframe F2 of a multi-bit signal is generated by adding and combining multiple microframes of 1-bit signals. As a result, one subframe F2 can include a multi-bit signal corresponding to the number of microframes in which incident light is detected within the subframe period T2.

[0056] In this way, a plurality of subframes F2 in which the period for acquiring the incident light differs from each other are acquired. This signal acquisition time can be associated with the distance from the distance image generating device to the distance measurement target. Then, the signal acquisition time at which the signal value is maximum can be determined from the distribution of the signal acquisition time and the signal values ​​of the plurality of subframes F2. Since it is estimated that the reflected light is incident on the imaging section 321 at the time when the signal value is maximum, the distance can be calculated by converting the signal acquisition time at which the signal value is maximum into the distance to the target X. Also, a distance image can be generated by calculating the distance for each pixel and acquiring a two-dimensional distribution of the distance.

[0057] 8, the length of a ranging frame period T1 required to acquire one ranging frame F1 depends on the number of subframes F2. Since the number of subframes F2 is a parameter corresponding to the number of ranging points, there is a trade-off between the distance resolution and the frame rate.

[0058] 9 shows a method for driving the distance image generating device in one distance measurement frame period T1. The driving method of this embodiment will be described with reference to the flowchart of FIG.

[0059] In the flowchart shown in Fig. 9, the processing from "START" to "END" indicates processing performed in a ranging frame period T1 in which one ranging frame F1 in Fig. 8 is acquired. One cycle of processing in the loop from step S11 to step S15 indicates processing performed in a sub-frame period T2 in which one sub-frame F2 in Fig. 8 is acquired. One cycle of processing in the loop from step S11 to step S13 indicates processing performed in a micro-frame period T3 in which one micro-frame F3 in Fig. 8 is acquired.

[0060] In step S11, the light source control unit 312 controls the pulse light source 311 to emit pulse light within a predetermined distance measurement range. In synchronization with this, the gate pulse generation unit 322 controls the imaging unit 321 to start detecting incident light by global gate driving.

[0061] In step S12, the microframe readout unit 323 reads out a microframe from the imaging unit 321 every time a microframe period elapses. The read out microframe is stored in the memory of the microframe adder 324. This memory has a storage capacity capable of storing multiple bits of data for each pixel. The microframe adder 324 sequentially adds the value of the microframe to the value stored in the memory every time a microframe is read out. In this way, the microframe adder 324 adds up multiple microframes in a subframe period to generate a subframe. The number of additions in the microframe adder 324 is controlled by the addition number control unit 325. For this control, the addition number control unit 325 holds information on the number of additions set in advance. In this way, the microframe readout unit 323 functions as an acquisition unit that acquires a microframe composed of a 1-bit signal based on the incident light to the photoelectric conversion element. The microframe adder 324 also functions as a synthesis unit that synthesizes multiple microframes acquired in different periods. For example, when the number of additions is 64, a subframe signal having 6-bit gradation can be generated by synthesizing 64 microframes.

[0062] In step S13, the microframe adder 324 determines whether or not the addition of a preset number of microframes has been completed. If the addition of the preset number of microframes has not been completed (NO in step S13), the process proceeds to step S11, where the next microframe is read. If the addition of the preset number of microframes has been completed (YES in step S13), the process proceeds to step S14.

[0063] In step S14, the subframe output unit 326 reads out the subframes for which the addition has been completed from the memory of the microframe adder unit 324, and outputs the subframes to the subframe group storage unit 331. The subframe group storage unit 331 stores the subframes output from the subframe output unit 326. The subframe group storage unit 331 is configured to be able to store multiple subframes used to generate one ranging frame individually for each subframe period.

[0064] In step S15, the signal processing circuit 33 judges whether the subframe group storage unit 331 has acquired a predetermined number of subframes (i.e., the number of ranging points). If acquisition of subframes for the number of ranging points has not been completed (NO in step S15), the process proceeds to step S11, and multiple microframes are acquired and added again to read the next subframe. In this case, the start time of global gate driving relative to the light emission time is shifted (gate shift) by one subframe period, and the same process is performed. If acquisition of subframes for the number of ranging points has been completed (YES in step S15), the process proceeds to step S16. Subframes for the number of ranging points are acquired by the loop between steps S11 and S15.

[0065] In step S16, distance image generating unit 332 acquires multiple subframes in one ranging frame period from subframe group storage unit 331. Distance image generating unit 332 calculates the subframe in which the signal value is maximum for each pixel and the distance corresponding to the subframe, thereby generating a distance image showing a two-dimensional distribution of distances. Then, distance image generating unit 332 outputs the distance image to a device external to signal processing circuit 33. This distance image can be used, for example, to detect the surrounding environment of the vehicle. Note that distance image generating unit 332 may store the distance image in a memory inside the distance image generating device.

[0066] In this embodiment, the pixels in the imaging section 321 are divided into four pixel groups. The exposure period (corresponding to step S11 in FIG. 9) for reading out the microframes differs for each pixel group. A specific example will be described with reference to FIG. 10(a) and FIG. 10(b). FIG. 10(a) is a schematic diagram showing the arrangement of pixel groups in a pixel array according to this embodiment, and FIG. 10(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0067] The pixel array of this embodiment includes a first pixel group 327A ("A" in FIG. 10(a)), a second pixel group 327B ("B" in FIG. 10(a)), a third pixel group 327C ("C" in FIG. 10(a)), and a fourth pixel group 327D ("D" in FIG. 10(a)). As shown in FIG. 10(a), the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D form a two-dimensional repeated array with four pixels forming one block.

[0068] "Light emission" in FIG. 10(b) indicates the light emission timing of the pulsed light source 311. As shown in FIG. 10(b), the pulsed light source 311 emits light at a constant cycle under the control of the light source control unit 312. This cycle corresponds to the length of one microframe period during which one microframe is acquired. G_A " to "P G_D " indicates the input timing of multiple types of gate pulses input from the gate pulse generating unit 322 to the imaging unit 321. G_A ", "P G_B ", "P G_C " and "P G_D" indicate different gate pulses for controlling the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D, respectively. The gate pulses synchronized with the light emission timing L01 corresponding to the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D are respectively gate pulses G01, G02, G03, and G04. The gate pulses G01, G02, G03, and G04 become high level after a predetermined time has elapsed since the light emission of the pulse light source 311, but the periods during which they become high level within one microframe period are different from each other. In other words, the gate pulses G01, G02, G03, and G04 become high level at the time when the first time, the second time, the third time, and the fourth time, which are different from each other, have elapsed since the light emission of the pulse light source 311. All of the first time to the fourth time are shorter than the length of the microframe period.

[0069] In this way, by varying the timing of the gate pulses G01, G02, G03, and G04, it is possible to vary the exposure time in each of the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D. Therefore, it is possible to measure four types of distance measurement points within one microframe period.

[0070] As described above, one subframe is generated by adding up a plurality of microframes. That is, the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D output signals for generating subframes of different distance measurement points. When one subframe is generated, at the start of the next subframe period, the timing of the gate pulse in the first pixel group 327A is gate shifted by a predetermined time interval from the timing of the gate pulse in the current subframe period. Similarly, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D are also gate shifted. By such gate shift, a predetermined period corresponding to each subframe period can be set as an exposure period.

[0071] In this way, by making the exposure time different for each of the multiple pixel groups and performing gate shifting for each of the multiple pixel groups for each subframe period, the number of gate shifts required to measure the required number of ranging points is reduced. This reduces the total period of the multiple subframes, i.e., the length of the frame period (T1 in FIG. 8). Therefore, it is possible to improve the frame rate without reducing the number of ranging points (distance resolution). As described above, according to this embodiment, a photoelectric conversion device is provided in which the frame rate is improved while maintaining appropriate distance resolution.

[0072] In this embodiment, the number of types of pixel groups is set to four, but the number of types of pixel groups may be at least two, and the effect of improving the frame rate can be obtained even if the number is other than four. Also, the number of photoelectric conversion elements included in one pixel group may be at least one. That is, for two photoelectric conversion elements (first photoelectric conversion element and second photoelectric conversion element), the above-mentioned effect can be obtained as long as the first exposure period of the first photoelectric conversion element and the second exposure period of the second photoelectric conversion element are different from each other in one microframe period. Also, the arrangement of the pixel groups is not limited to that shown in FIG. 10(a) and can be changed as appropriate.

[0073] In this embodiment, although the frame rate can be improved by arranging multiple pixel groups, the in-plane resolution of the ranging image may be reduced. Therefore, a method may be further applied in which pixels with missing information in the ranging image are supplemented with surrounding pixels to reduce the effect on the in-plane resolution.

[0074] [Second embodiment] In this embodiment, another example of the arrangement of the pixel group and the timing of the gate pulses described in the first embodiment will be described. Descriptions of elements common to the first embodiment may be omitted or simplified as appropriate.

[0075] FIG. 11(a) is a schematic diagram showing an arrangement of a pixel group in a pixel array according to this embodiment, and FIG. 11(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0076] The pixel array of this embodiment includes a first pixel group 328A ("A" in FIG. 11(a)), a second pixel group 328B ("B" in FIG. 11(a)), a third pixel group 328C ("C" in FIG. 11(a)), and a fourth pixel group 328D ("D" in FIG. 11(a)). The first pixel group 328A and the second pixel group 328B (first photoelectric conversion element) are configured to have sensitivity to light of a first wavelength. The third pixel group 328C and the fourth pixel group 328D (second photoelectric conversion element) are configured to have sensitivity to light of a second wavelength different from the first wavelength. More specifically, a first color filter that transmits light of the first wavelength is arranged in the first pixel group 328A and the second pixel group 328B, and a second color filter that transmits light of the second wavelength is arranged in the third pixel group 328C and the fourth pixel group 328D. In this embodiment, the first wavelength and the second wavelength range may both be, for example, in the infrared region.

[0077] The pulsed light source 311 of this embodiment is configured to be able to emit light of the first wavelength and light of the second wavelength individually with different cycles. "Emission (first wavelength)" in FIG. 11(b) indicates the timing of emission of light of the first wavelength by the pulsed light source 311. "Emission (second wavelength)" in FIG. 11(b) indicates the timing of emission of light of the second wavelength by the pulsed light source 311. As shown in FIG. 11(b), the light of the first wavelength and the light of the second wavelength have different emission cycle lengths. In the example of FIG. 11(b), the length of the emission cycle of the light of the second wavelength is twice the length of the emission cycle of the light of the first wavelength.

[0078] For the first pixel group 328A and the second pixel group 328B, gate pulses G05 and G06 are input at a timing synchronized with the emission timing L02 of the light of the first wavelength. For the third pixel group 328C and the fourth pixel group 328D, gate pulses G07 and G08 are input at a timing delayed by one period of the emission of the first wavelength from the emission timing L03 of the light of the second wavelength. Therefore, the acquisition period of the microframes for the first pixel group 328A and the second pixel group 328B is different from the acquisition period of the microframes for the third pixel group 328C and the fourth pixel group 328D. By setting in this way, the first pixel group 328A and the second pixel group 328B are used as pixel groups for distance measurement of a short distance (first distance range), and the third pixel group 328C and the fourth pixel group 328D are used as pixel groups for distance measurement of a long distance (second distance range). This makes it possible to measure a plurality of different distance measurement ranges within the same subframe period. In addition, in general, in methods that repeatedly acquire and add microframes of 1-bit signals, the distance measurement range is limited by the repetition period of the light emission pulse, but the method of this embodiment can acquire signals from both short and long distances, making it possible to measure distances over a wide range.

[0079] As described above, according to this embodiment, in addition to obtaining the same effects as the first embodiment, a photoelectric conversion device is provided that can simultaneously acquire multiple different ranging ranges without reducing the frame rate.

[0080] In this embodiment, the bit depth of the signal of the subframe obtained from the pixel group for short distance measurement and the bit depth of the signal of the subframe obtained from the pixel group for long distance measurement may be different from each other. When the number of additions of the signal for short distance measurement is 64 times, a subframe for short distance with 6 bit depth is obtained. In this case, since the number of times of emission of the light of the second wavelength is half the number of times of emission of the light of the first wavelength, the number of times of addition of the signal for long distance measurement is 32 times at most. Therefore, the subframe for long distance has a 5 bit depth. In this way, when the bit depths of the two signals are different, the bit depth may be adjusted.

[0081] In this embodiment, the pulse light source 311 emits light of two different wavelengths, but it may have three or more wavelengths. The ratio of the periods of the light of different wavelengths is not limited to two times, and can be set appropriately.

[0082] [Third embodiment] In this embodiment, a further example of the arrangement of the pixel group and the timing of the gate pulses described in the first and second embodiments will be described. Descriptions of elements common to the first and second embodiments may be omitted or simplified as appropriate.

[0083] FIG. 12(a) is a schematic diagram showing an arrangement of a pixel group in a pixel array according to this embodiment, and FIG. 12(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0084] The pixel array of this embodiment includes a first pixel group 329R ("R" in FIG. 12(a)), a second pixel group 329G ("G" in FIG. 12(a)), a third pixel group 329B ("B" in FIG. 12(a)), and a fourth pixel group 329Z ("Z" in FIG. 12(a)). The first pixel group 329R, the second pixel group 329G, the third pixel group 329B, and the fourth pixel group 329Z are configured to have sensitivity to light of different wavelengths. More specifically, a first color filter that transmits red light of a wavelength of 700 nm (first wavelength) is arranged in the first pixel group 329R (first photoelectric conversion element). A second color filter that transmits green light of a wavelength of 550 nm (third wavelength) is arranged in the second pixel group 329G (third photoelectric conversion element). The third pixel group 329B (fourth photoelectric conversion element) is provided with a third color filter that transmits blue light with a wavelength of 430 nm (fourth wavelength). The fourth pixel group 329Z (second photoelectric conversion element) is provided with a fourth color filter that transmits infrared light with a wavelength of 1000 nm (second wavelength).

[0085] The pulsed light source 311 of this embodiment is configured to emit light of an infrared wavelength. "Light emission (infrared wavelength)" in FIG. 12(b) indicates the timing of light emission of an infrared wavelength by the pulsed light source 311.G_Z " indicates a gate pulse for controlling the fourth pixel group 329Z. G_RGB " indicates a control gate pulse common to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B.

[0086] A gate pulse G09 is input to the fourth pixel group 329Z at a timing synchronized with the emission timing L04 of the infrared wavelength light. A gate pulse G10 is input to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B for a predetermined exposure period during which imaging is performed.

[0087] As a result, signals including information on the colors red, green, and blue are obtained in the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B, and a color image can be generated using these signals. Note that the three colors red, green, and blue are merely examples, and a color image can be generated as long as the acquired signals include information on multiple colors in the visible range. Also, as in the first and second embodiments, distance information is obtained in the fourth pixel group 329Z by inputting a gate pulse G09 at a timing synchronized with light emission.

[0088] As described above, according to the configuration of this embodiment, it is possible to obtain a signal for distance measurement and a signal for generating a color image within the same subframe period, thereby providing a photoelectric conversion device that can obtain a color image and a distance image while maintaining the distance resolution and frame rate.

[0089] [Fourth embodiment] In this embodiment, an example of a configuration in which a recharge pulse for resetting a photoelectric conversion element can be input to a pixel circuit in addition to the configuration of the third embodiment will be described. Descriptions of elements common to the first to third embodiments may be omitted or simplified as appropriate.

[0090] 13 is a schematic block diagram of a pixel according to this embodiment. In FIG. 13, a transistor 217 is arranged as a specific example of the quench element 202 in FIG. 5. The transistor 217 is an NMOS transistor. The source of the transistor 217 is connected to a connection node between the APD 201 and the input terminal of the waveform shaping unit 210. The voltage VH is supplied to the drain of the transistor 217. A recharge pulse is input to the gate of the transistor 217 from the vertical scanning circuit 110 via a drive line 218. When a high-level recharge pulse is input to the transistor 217, the transistor 217 is turned on, and the cathode potential of the APD 201 is reset to a potential at which avalanche multiplication is possible. This operation is called a recharge operation.

[0091] FIG. 14 is a drive timing diagram showing the timing of gate pulses and recharge pulses according to this embodiment. The configuration of the pixel group in the pixel array of this embodiment is the same as that of the third embodiment. Also, the pulse light source 311 of this embodiment is configured to be able to emit light of an infrared wavelength, similar to the third embodiment. R_Z " indicates a recharge pulse for controlling the fourth pixel group 329Z. R_RGB " indicates a control recharge pulse common to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B.

[0092] The recharge pulse synchronized with the light emission timing L05 and corresponding to the fourth pixel group 329Z is referred to as recharge pulse R01. The recharge pulse synchronized with the light emission timing L05 and corresponding to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B is referred to as recharge pulse R02. The recharge pulses R01 and R02 go high after a predetermined time has elapsed since the light emission of the pulsed light source 311, but the periods during which they are at high level within one microframe period are different from each other.

[0093] After the recharge pulse R01 falls, the gate pulse G11 corresponding to the fourth pixel group 329Z goes high. After the recharge pulse R02 falls, the gate pulse G12 corresponding to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B goes high.

[0094] In the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B, the period from the falling edge of the recharge pulse R02 to the falling edge of the gate pulse G12 is the photon detection period for capturing a color image. That is, in the configuration of this embodiment, the timing of the recharge pulse R02 and the gate pulse G12 is appropriately set, so that the charge accumulation time in capturing a color image can be controlled. In the configuration of the third embodiment, since the gate pulse G10 for capturing a color image is maintained at a high level, when the subject is a moving object, image blurring or mixing of signals from different subjects may occur. However, in the configuration of this embodiment, the influence of these factors can be reduced by appropriately setting the charge accumulation time in capturing a color image. Therefore, according to this embodiment, in addition to obtaining the same effect as the third embodiment, the image quality can be further improved.

[0095] [Fifth embodiment] In this embodiment, an example of a configuration capable of generating an external light map from microframes of multiple pixel groups in addition to the configurations of the first to fourth embodiments will be described. Descriptions of elements common to the first to fourth embodiments may be omitted or simplified as appropriate.

[0096] Fig. 15 is a functional block diagram showing an example of a schematic configuration of a distance measuring device according to this embodiment. In Fig. 15, in addition to the configuration of Fig. 7, a flag calculation unit 327 is further provided in the light receiving device 32, and a map generation unit 333 is further provided in the signal processing circuit 33. The other configurations are the same as those of any of the first to fourth embodiments. Below, the operations of the flag calculation unit 327 and the map generation unit 333 of this embodiment will be described on the premise of the configuration of the first embodiment.

[0097] The flag calculation unit 327 performs logical operations such as logical AND on the 1-bit signals for microframes output from the multiple pixel groups, and outputs the result as an external light flag. The external light flag is a 1-bit signal that indicates the presence or absence of ambient light (external light) emitted from sources other than the light emitting device 31. For example, the flag calculation unit 327 calculates the logical AND of four 1-bit signals output from the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D. In this case, when the logical values ​​of the four 1-bit signals are all "1", the logical value of the calculation result is "1". The pixel group to be added can be selected from multiple adjacent pixels, such as the four pixels marked with symbols in FIG. 10.

[0098] The flag calculation unit 327 generates multiple external light flags by performing similar logical calculation processing on each of multiple regions selected from the entire pixel array. The map generation unit 333 acquires the multiple external light flags generated by the flag calculation unit 327, generates an external light map in which the positions of the pixel array are associated with the logical values ​​of the external light flags, and outputs the map to the outside.

[0099] Fig. 16 is a schematic diagram showing an example of an external light map according to this embodiment. Fig. 16 is an image in which a box with "1" added only to the area where the logical value of the external light flag is "1" is superimposed on an image captured by a photoelectric conversion device mounted on a vehicle. As shown in Fig. 16, the logical value of the external light flag is "1" in the area of ​​blue sky, which is generally bright due to the influence of sunlight.

[0100] In general, in a method of measuring distance by emitting light from a light source and detecting the light reflected by an object, external light such as sunlight that exists independently of the light from the light source can affect the accuracy. For example, external light that is not reflected from the object may produce a false signal indicating the object.

[0101] The detection value based on the external light often depends on the position. For example, the area receiving the external light from the blue sky as shown in FIG. 16 spreads over a wide area at the top of the image. In this embodiment, the external light flag is calculated by logical operation of 1-bit signals output from multiple pixels, so that it is possible to detect pixels into which the above-mentioned fixed tendency of external light is likely to be incident. By generating an external light map using this external light flag, it is possible to output information indicating the area into which the external light is likely to be incident to the outside. The external light map can be used in the image processing at the subsequent stage. For example, when performing processing of object recognition in an image, by excluding the area with a logical value of "1" in the external light map from the processing, it is not necessary to spend processing resources on the area with low accuracy due to the external light, so that the processing can be speeded up. This object recognition can be applied to, for example, automatic driving, etc.

[0102] As described above, according to this embodiment, a photoelectric conversion device capable of outputting information indicating the accuracy of a signal to the outside is provided.

[0103] In this embodiment, the logical AND is used as an example of the logical operation performed by the flag calculation unit 327, but the logical operation is not limited to this and may be any process that outputs a value based on signals from multiple pixels. For example, the logical operation may be a logical OR.

[0104] [Sixth embodiment] 17(a) and 17(b) are block diagrams of devices related to the vehicle-mounted distance measuring device in this embodiment. The device 80 has a distance measuring unit 803, which is an example of the distance image generating device in the above-mentioned embodiment, and a signal processing device (processing device) that processes a signal from the distance measuring unit 803. The device 80 has a distance measuring unit 803 that measures the distance to an object, and a collision determination unit 804 that determines whether or not there is a possibility of collision based on the measured distance. Here, the distance measuring unit 803 is an example of a distance information acquisition means that acquires distance information to the object. That is, the distance information is information related to the distance to the object, etc. The collision determination unit 804 may use the distance information to determine the possibility of collision.

[0105] The device 80 is connected to a vehicle information acquisition device 810, and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. In addition, the device 80 is connected to a control ECU 820, which is a control device that outputs a control signal to generate a braking force for the vehicle based on the judgment result of the collision judgment unit 804. In addition, the device 80 is also connected to an alarm device 830 that issues an alarm to the driver based on the judgment result of the collision judgment unit 804. For example, when the judgment result of the collision judgment unit 804 indicates that there is a high possibility of a collision, the control ECU 820 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 830 warns the user by sounding an alarm such as a sound, displaying alarm information on the screen of a car navigation system, etc., and applying vibrations to a seat belt or steering wheel. These devices of the device 80 function as a mobile object control unit that controls the operation of controlling the vehicle as described above.

[0106] In this embodiment, the device 80 measures distances around the vehicle, for example, in front or behind. Fig. 17(b) shows the device when measuring distances in front of the vehicle (distance measurement range 850). A vehicle information acquisition device 810 as a distance measurement control means sends an instruction to the device 80 or distance measurement unit 803 to perform a distance measurement operation. With this configuration, the accuracy of distance measurement can be further improved.

[0107] Although the above describes an example of control to prevent collision with other vehicles, the present invention can also be applied to control of automatic driving by following other vehicles, control of automatic driving to prevent deviation from lanes, etc. Furthermore, the device is not limited to vehicles such as automobiles, but can be applied to moving bodies (moving devices) such as ships, aircraft, artificial satellites, industrial robots, and consumer robots. In addition, the present invention can be applied to devices that use object recognition or biometric recognition, such as intelligent transport systems (ITS) and surveillance systems, in addition to moving bodies.

[0108] [Modified embodiment] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.

[0109] The disclosure of this specification includes the complement of the concepts described in this specification. In other words, if the specification states, for example, that "A is B" (A=B), the specification is deemed to disclose or suggest that "A is not B" even if the statement that "A is not B" (A≠B) is omitted. This is because when it states that "A is B," it is assumed that the case that "A is not B" is taken into consideration.

[0110] The disclosure of this specification includes the following configurations. (Configuration 1) A plurality of photoelectric conversion elements; an acquisition unit that acquires a microframe configured by a 1-bit signal based on incident light to each of the plurality of photoelectric conversion elements; a synthesis unit for synthesizing a plurality of the microframes to generate a subframe constituted by a multi-bit signal; having the plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element, In an acquisition period of one of the microframes, a first exposure period during which the 1-bit signal is generated based on the light incident on the first photoelectric conversion element and a second exposure period during which the 1-bit signal is generated based on the light incident on the second photoelectric conversion element are different from each other. A photoelectric conversion device comprising: (Configuration 2) the first photoelectric conversion element is sensitive to light of a first wavelength; The second photoelectric conversion element has sensitivity to light having a second wavelength different from the first wavelength. 2. The photoelectric conversion device according to configuration 1. (Configuration 3) An acquisition cycle of the microframes based on the light incident on the first photoelectric conversion element and an acquisition cycle of the microframes based on the light incident on the second photoelectric conversion element are different from each other. 3. The photoelectric conversion device according to configuration 2. (Configuration 4) a signal based on the light incident on the first photoelectric conversion element is used for measuring a distance in a first distance range; A signal based on the light incident on the second photoelectric conversion element is used for measuring a distance in a second distance range different from the first distance range. 4. The photoelectric conversion device according to claim 2 or 3. (Configuration 5) a signal based on the light incident on the first photoelectric conversion element is used to generate an image; A signal based on the incident light to the second photoelectric conversion element is used for distance measurement. 3. The photoelectric conversion device according to configuration 2. (Configuration 6) the plurality of photoelectric conversion elements further include a third photoelectric conversion element and a fourth photoelectric conversion element, the third photoelectric conversion element is sensitive to light having a third wavelength different from both the first wavelength and the second wavelength; The fourth photoelectric conversion element has sensitivity to light having a fourth wavelength different from any of the first wavelength, the second wavelength, and the third wavelength. 6. The photoelectric conversion device according to configuration 5. (Configuration 7) A signal based on the incident light to the first photoelectric conversion element, the third photoelectric conversion element, and the fourth photoelectric conversion element is used to generate a color image. 7. The photoelectric conversion device according to configuration 6. (Configuration 8) the first wavelength is in the visible range, The second wavelength is in the infrared region. 8. The photoelectric conversion device according to any one of configurations 5 to 7. (Configuration 9) In an acquisition period of one of the microframes, the time at which the first photoelectric conversion element is reset and the time at which the second photoelectric conversion element is reset are different from each other. 9. The photoelectric conversion device according to any one of configurations 1 to 8. (Configuration 10) a calculation unit that generates a signal by logical calculation based on the 1-bit signal based on the light incident on the first photoelectric conversion element and the 1-bit signal based on the light incident on the second photoelectric conversion element. 10. The photoelectric conversion device according to any one of configurations 1 to 9. (Configuration 11) The signal output by the computing unit indicates the presence or absence of ambient light. 11. The photoelectric conversion device according to configuration 10. (Configuration 12) The logical operation is a logical AND of the 1-bit signal based on the light incident on the first photoelectric conversion element and the 1-bit signal based on the light incident on the second photoelectric conversion element. 12. The photoelectric conversion device according to claim 10 or 11. (Configuration 13) the first exposure period starts when a first time has elapsed since a light source emits light toward an object of distance measurement; The second exposure period starts when a second time different from the first time has elapsed from the light emission timing. 13. The photoelectric conversion device according to any one of configurations 1 to 12. (Configuration 14) the photoelectric conversion element includes an avalanche photodiode, The 1-bit signal indicates whether or not a photon is incident on the avalanche photodiode during the period in which the microframe is acquired. 14. The photoelectric conversion device according to any one of configurations 1 to 13. (Configuration 15) The synthesis unit generates the multi-bit signal by adding up the value of the 1-bit signal each time the microframe is acquired. 15. The photoelectric conversion device according to any one of configurations 1 to 14. (Configuration 16) The photoelectric conversion device according to any one of configurations 1 to 15, a distance image generating unit that generates a distance image based on a signal generated by the photoelectric conversion device; A distance image generating device comprising: (Configuration 17) A mobile object, The photoelectric conversion device according to any one of configurations 1 to 15, a moving object control unit that controls the moving object based on distance information acquired by the photoelectric conversion device; A moving object comprising:

[0111] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0112] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0113] 321 Imaging unit 322 Gate pulse generator 323 Microframe Readout Unit 324 Microframe Addition Unit

Claims

1. A plurality of photoelectric conversion elements; an acquisition unit that acquires a microframe configured by a 1-bit signal based on light incident on each of the plurality of photoelectric conversion elements; a synthesizing unit that synthesizes a plurality of the microframes to generate a subframe that is configured by a multi-bit signal; a distance image generating unit that generates a distance measurement frame using a plurality of the subframes; and the plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element, In an acquisition period of one microframe, a first exposure period in which the 1-bit signal is generated based on the incident light to the first photoelectric conversion element and a second exposure period in which the 1-bit signal is generated based on the incident light to the second photoelectric conversion element are different from each other. A photoelectric conversion device characterized by:

2. the first photoelectric conversion element is sensitive to light of a first wavelength; The second photoelectric conversion element has sensitivity to light of a second wavelength different from the first wavelength.

2. The photoelectric conversion device according to claim 1.

3. an acquisition cycle of the microframes based on the incident light to the first photoelectric conversion element and an acquisition cycle of the microframes based on the incident light to the second photoelectric conversion element are different from each other; 3. The photoelectric conversion device according to claim 2.

4. a signal based on the incident light to the first photoelectric conversion element is used for distance measurement in a first distance range; A signal based on the incident light to the second photoelectric conversion element is used for measuring distance in a second distance range different from the first distance range.

3. The photoelectric conversion device according to claim 2.

5. a signal based on the incident light to the first photoelectric conversion element is used to generate an image; A signal based on the incident light to the second photoelectric conversion element is used for distance measurement.

3. The photoelectric conversion device according to claim 2.

6. the plurality of photoelectric conversion elements further include a third photoelectric conversion element and a fourth photoelectric conversion element, the third photoelectric conversion element is sensitive to light of a third wavelength different from both the first wavelength and the second wavelength; the fourth photoelectric conversion element is sensitive to light of a fourth wavelength different from any of the first wavelength, the second wavelength, and the third wavelength; 6. The photoelectric conversion device according to claim 5.

7. Signals based on the incident light to the first photoelectric conversion element, the third photoelectric conversion element, and the fourth photoelectric conversion element are used to generate a color image.

7. The photoelectric conversion device according to claim 6.

8. the first wavelength is in the visible range, The second wavelength is a wavelength in the infrared region.

6. The photoelectric conversion device according to claim 5.

9. In one acquisition period of the microframe, the time at which the first photoelectric conversion element is reset and the time at which the second photoelectric conversion element is reset are different from each other.

2. The photoelectric conversion device according to claim 1.

10. a calculation unit that generates a signal by logical calculation based on the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element; 2. The photoelectric conversion device according to claim 1.

11. The signal output by the calculation unit indicates the presence or absence of ambient light.

11. The photoelectric conversion device according to claim 10.

12. The logical operation is a logical product of the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element.

11. The photoelectric conversion device according to claim 10.

13. the first exposure period starts when a first time has elapsed since a light source emits light toward an object of distance measurement; The second exposure period starts when a second time different from the first time has elapsed from the light emission timing.

2. The photoelectric conversion device according to claim 1.

14. the photoelectric conversion element includes an avalanche photodiode, The one-bit signal indicates whether or not a photon is incident on the avalanche photodiode within the period during which the microframe is acquired.

2. The photoelectric conversion device according to claim 1.

15. The synthesizing unit generates the multi-bit signal by adding up the value of the 1-bit signal each time the microframe is acquired.

2. The photoelectric conversion device according to claim 1.

16. A mobile object, The photoelectric conversion device according to any one of claims 1 to 15, a moving object control unit that controls the moving object based on distance information acquired by the photoelectric conversion device; A moving object comprising:

Images