Photoelectric converter

Abstract

To provide a photoelectric conversion device that improves a frame rate while ensuring appropriate distance resolution.SOLUTION: A photoelectric conversion device has: a plurality of photoelectric conversion elements; an acquisition unit that acquires a micro-frame comprising one-bit signals on the basis of incident light on each of the plurality of photoelectric conversion elements; and a composition unit that combines the plurality of micro-frames to create a sub-frame comprising multi-bit signals. The plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element. In a period for acquiring the one micro-frame, a first exposure time during which creation of the one-bit signals based on the incident light on the first photoelectric conversion element, and a second exposure time during which creation of the one-bit signals based on the incident light on the second photoelectric conversion element, are different from each other.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] This invention relates to a photoelectric conversion device. [Background technology]

[0002] Patent Document 1 discloses a distance measuring device that measures the distance to an object by emitting light from a light source and receiving the light, including reflected light from the object, with a photodetector. In the distance measuring device of Patent Document 1, a SPAD (Single Photon Avalanche Diode) element is used as the photodetector, which acquires a signal by multiplying electrons generated by photoelectric conversion. Patent Document 1 discloses a method for repeatedly performing measurements while changing the gating period during which photon detection takes place in the SPAD element. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] U.S. Patent Application Publication No. 2017 / 0052065 [Overview of the project] [Problems that the invention aims to solve]

[0004] In distance measurement methods such as those disclosed in Patent Document 1, there is a trade-off between distance resolution and frame rate. However, in order to improve distance measurement performance, it is sometimes necessary to achieve both an appropriate distance resolution and an improved frame rate.

[0005] The present invention aims to provide a photoelectric converter that improves the frame rate while ensuring appropriate distance resolution. [Means for solving the problem]

[0006] According to one disclosure of this specification, a plurality of photoelectric conversion elements, an acquisition unit that acquires a microframe composed of a 1-bit signal based on the light incident on each of the plurality of photoelectric conversion elements, and a synthesis unit that generates a subframe composed of a multi-bit signal by synthesizing the plurality of microframes, A distance image generation unit that generates a distance measurement frame using a plurality of the aforementioned subframes, A photoelectric conversion device is provided, wherein the plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element, and in the acquisition period of one microframe, a first exposure period in which the 1-bit signal is generated based on the incident light to the first photoelectric conversion element and a second exposure period in which the 1-bit signal is generated based on the incident light to the second photoelectric conversion element are different from each other. [Effects of the Invention]

[0007] According to the present invention, a photoelectric converter is provided that improves the frame rate while ensuring appropriate distance resolution. [Brief explanation of the drawing]

[0008] [Figure 1] This is a hardware block diagram showing a schematic configuration example of a distance measuring device according to the first embodiment. [Figure 2] This is a schematic diagram showing the overall configuration of the photoelectric conversion device according to the first embodiment. [Figure 3] This is a schematic block diagram showing an example of the configuration of a sensor substrate according to the first embodiment. [Figure 4] This is a schematic block diagram showing an example of the configuration of a circuit board according to the first embodiment. [Figure 5] This is a schematic block diagram showing an example of the configuration of one pixel in the photoelectric conversion unit and pixel signal processing unit according to the first embodiment. [Figure 6] This is a diagram illustrating the operation of an avalanche photodiode according to the first embodiment. [Figure 7] This is a functional block diagram showing a schematic configuration example of a distance measuring device according to the first embodiment. [Figure 8]It is a schematic diagram for explaining the ranging frame, sub-frame, and micro-frame according to the first embodiment. [Figure 9] It is a flowchart showing the operation in one ranging frame period of the distance image generation device according to the first embodiment. [Figure 10] It is a schematic diagram and a driving timing diagram of the pixel array according to the first embodiment. [Figure 11] It is a schematic diagram and a driving timing diagram of the pixel array according to the second embodiment. [Figure 12] It is a schematic diagram and a driving timing diagram of the pixel array according to the third embodiment. <着 [Figure 13] It is a schematic block diagram of a pixel according to the fourth embodiment. [Figure 14] It is a driving timing diagram according to the fourth embodiment. [Figure 15] It is a functional block diagram showing a schematic configuration example of the ranging device according to the fifth embodiment. [Figure 16] It is a schematic diagram showing an example of an external light map according to the fifth embodiment. [Figure 17] It is a schematic diagram of a device according to the sixth embodiment.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described while referring to the drawings. The same elements or corresponding elements are given common reference numerals throughout the plurality of drawings, and the description thereof may be omitted or simplified. <00000t95> [First Embodiment] FIG. 1 is a hardware block diagram showing a schematic configuration example of a distance image generation device 30 according to the present embodiment. The distance image generation device 30 includes a light emitting device 31, a light receiving device 32, and a signal processing circuit 33. Note that the configuration of the distance image generation device 30 shown in the present embodiment is an example and is not limited to the illustrated configuration.

[0011] The distance image generation device 30 is a device that measures the distance to an object X using technologies such as LiDAR (Light Detection and Ranging). The distance image generation device 30 measures the distance from the device 30 to the object X based on the time difference between when light emitted from the light emitter 31 is reflected by the object X and when it is received by the light receiver 32. In addition, the distance image generation device 30 can measure distances at multiple points in two dimensions by emitting laser light into a predetermined distance measurement range including the object X and receiving the reflected light with a pixel array. As a result, the distance image generation device 30 can generate and output a distance image.

[0012] The light received by the light receiving device 32 includes ambient light such as sunlight in addition to reflected light from the object X. Therefore, the distance image generating device 30 measures the incident light during each of several periods (bin periods) and uses a method that reduces the influence of ambient light by determining that reflected light was incident during the period when the light intensity is at its peak.

[0013] The light-emitting device 31 is a device that emits light, such as laser light, to the outside of the distance image generation device 30. The signal processing circuit 33 may include a processor that performs arithmetic processing on digital signals, a memory that stores digital signals, etc. The memory may be, for example, a semiconductor memory.

[0014] The light-receiving device 32 generates a pulse signal that includes a pulse based on the incident light. The light-receiving device 32 is, for example, a photoelectric converter that includes an avalanche photodiode as a photoelectric conversion element. In this case, when one photon is incident on the avalanche photodiode and a charge is generated, one pulse is generated by avalanche multiplication. However, the light-receiving device 32 may also use a photoelectric conversion element that uses, for example, another photodiode.

[0015] In this embodiment, the light receiving device 32 includes a pixel array in which a plurality of photoelectric conversion elements (pixels) are arranged in a plurality of rows and a plurality of columns. Here, a specific example of the configuration of the photoelectric conversion device, which is a photoelectric conversion device, will be described with reference to Figures 2 to 6. The example of the configuration of the photoelectric conversion device described below is just one example. The photoelectric conversion device applicable to the light receiving device 32 is not limited to this, and any device that can realize the function shown in Figure 7, which will be described later, is acceptable.

[0016] Figure 2 is a schematic diagram showing the overall configuration of the photoelectric converter 100 according to this embodiment. The photoelectric converter 100 has a sensor substrate 11 (first substrate) and a circuit board 21 (second substrate) stacked on top of each other. The sensor substrate 11 and the circuit board 21 are electrically interconnected. The sensor substrate 11 has a pixel region 12 on which a plurality of pixels 101 are arranged in a plurality of rows and a plurality of columns. The circuit board 21 has a first circuit region 22 on which a plurality of pixel signal processing units 103 are arranged in a plurality of rows and a plurality of columns, and a second circuit region 23 arranged on the outer periphery of the first circuit region 22. The second circuit region 23 may include circuits for controlling the plurality of pixel signal processing units 103. The sensor substrate 11 has a light incident surface that receives incident light and a connection surface that faces the light incident surface. The sensor substrate 11 is connected to the circuit board 21 on the connection surface side. In other words, the photoelectric converter 100 is a so-called back-illuminated type.

[0017] In this specification, "plan view" refers to viewing from a direction perpendicular to the surface opposite to the light incidence surface. Similarly, "cross-section" refers to the surface of the sensor substrate 11 perpendicular to the surface opposite to the light incidence surface. While the light incidence surface may appear rough at a microscopic level, in such cases, the plan view is defined based on the light incidence surface as viewed macroscopically.

[0018] In the following description, the sensor substrate 11 and the circuit board 21 are assumed to be diced chips, but the sensor substrate 11 and the circuit board 21 are not limited to chips. For example, the sensor substrate 11 and the circuit board 21 may be wafers. Furthermore, if the sensor substrate 11 and the circuit board 21 are diced chips, the photoelectric converter 100 may be manufactured by stacking wafers and then dicing them, or by stacking wafers after dicing them.

[0019] Figure 3 is a schematic block diagram showing an example of the arrangement of the sensor substrate 11. The pixel region 12 contains multiple pixels 101 arranged in multiple rows and multiple columns. Each of the multiple pixels 101 has a photoelectric conversion unit 102 on the substrate, which includes an avalanche photodiode (hereinafter referred to as APD) as a photoelectric conversion element.

[0020] In an APD (Automated Precipitator), the conductivity type of the charge used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type in which the majority carriers are charges of the same polarity as the signal charge. Conversely, the conductivity type opposite to the first conductivity type, i.e., a conductivity type in which the majority carriers are charges of a different polarity than the signal charge, is called the second conductivity type. In the APD described below, the anode of the APD is at a fixed potential, and the signal is extracted from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is the N-type semiconductor region, and the semiconductor region of the second conductivity type is the P-type semiconductor region. Alternatively, the cathode of the APD may be at a fixed potential, and the signal may be extracted from the anode of the APD. In this case, the semiconductor region of the first conductivity type is the P-type semiconductor region, and the semiconductor region of the second conductivity type is the N-type semiconductor region. Furthermore, the following description focuses on the case where one node of the APD is at a fixed potential, but a configuration in which the potentials of both nodes fluctuate is also possible.

[0021] Figure 4 is a schematic block diagram showing an example of the configuration of the circuit board 21. The circuit board 21 has a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged in a plurality of rows and a plurality of columns.

[0022] Furthermore, the circuit board 21 includes a vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, a pixel output signal line 113, an output circuit 114, and a control signal generation unit 115. The multiple photoelectric conversion units 102 shown in Figure 3 and the multiple pixel signal processing units 103 shown in Figure 4 are electrically connected via connecting wiring provided for each pixel 101.

[0023] The control signal generation unit 115 is a control circuit that generates and supplies control signals to drive the vertical scanning circuit 110, the horizontal scanning circuit 111, and the readout circuit 112. In this way, the control signal generation unit 115 controls the drive timing and other aspects of each component.

[0024] The vertical scanning circuit 110 supplies control signals to each of the multiple pixel signal processing units 103 based on the control signals supplied from the control signal generation unit 115. The vertical scanning circuit 110 supplies control signals to each pixel signal processing unit 103 row by row via drive lines provided for each row of the first circuit region 22. As will be described later, there may be multiple drive lines for each row. Logic circuits such as shift registers and address decoders may be used in the vertical scanning circuit 110. This allows the vertical scanning circuit 110 to select the row from which the pixel signal processing unit 103 will output a signal.

[0025] The signal output from the photoelectric conversion unit 102 of the pixel 101 is processed by the pixel signal processing unit 103. The pixel signal processing unit 103 acquires and stores a digital signal having multiple bits by counting the number of pulses output from the APD included in the photoelectric conversion unit 102.

[0026] The pixel signal processing unit 103 does not necessarily have to be provided for every pixel 101. For example, one pixel signal processing unit 103 may be shared by multiple pixels 101. In this case, the pixel signal processing unit 103 provides signal processing functionality to each pixel 101 by sequentially processing the signals output from each photoelectric conversion unit 102.

[0027] The horizontal scanning circuit 111 supplies control signals to the readout circuit 112 based on control signals supplied from the control signal generation unit 115. The pixel signal processing unit 103 is connected to the readout circuit 112 via pixel output signal lines 113, which are provided for each column of the first circuit region 22. The pixel output signal line 113 of one column is shared by multiple pixel signal processing units 103 of the corresponding column. The pixel output signal line 113 includes multiple wires and has at least the function of outputting a digital signal from each pixel signal processing unit 103 to the readout circuit 112, and the function of supplying a control signal to the pixel signal processing unit 103 for selecting the column from which to output a signal. The readout circuit 112 outputs a signal to an external storage unit or signal processing unit of the photoelectric converter 100 via the output circuit 114 based on the control signals supplied from the control signal generation unit 115.

[0028] The photoelectric conversion units 102 in the pixel region 12 may be arranged in a one-dimensional manner. Furthermore, the pixel signal processing unit 103 does not necessarily have to be provided for every pixel 101. For example, one pixel signal processing unit 103 may be shared by multiple pixels 101. In this case, the pixel signal processing unit 103 provides signal processing functionality to each pixel 101 by sequentially processing the signals output from each photoelectric conversion unit 102.

[0029] As shown in Figures 3 and 4, a first circuit region 22, in which multiple pixel signal processing units 103 are arranged, is located in the region that overlaps with the pixel region 12 in a plan view. A vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generation unit 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. A second circuit region 23 (mentioned above in Figure 2) is located in the region that overlaps with the non-pixel region in a plan view, and in which the vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generation unit 115 are arranged.

[0030] Note that the arrangement of the pixel output signal lines 113, the readout circuit 112, and the output circuit 114 is not limited to those shown in Figure 3. For example, the pixel output signal lines 113 may be arranged to extend in the row direction and be shared by multiple pixel signal processing units 103 in the corresponding row. The readout circuit 112 may be arranged so that the pixel output signal lines 113 of each row are connected.

[0031] Figure 5 is a schematic block diagram showing an example of the configuration of one pixel of the photoelectric conversion unit 102 and the pixel signal processing unit 103 according to this embodiment. Figure 5 schematically shows a more specific configuration example, including the connection relationship between the photoelectric conversion unit 102 arranged on the sensor substrate 11 and the pixel signal processing unit 103 arranged on the circuit board 21. In Figure 5, the drive lines between the vertical scanning circuit 110 and the pixel signal processing unit 103 in Figure 4 are shown as drive lines 213, 214, and 215.

[0032] The photoelectric conversion unit 102 has an APD 201. The pixel signal processing unit 103 has a quench element 202, a waveform shaping unit 210, a counter circuit 211, a selection circuit 212, and a gating circuit 216. Note that the pixel signal processing unit 103 only needs to have at least one of the waveform shaping unit 210, the counter circuit 211, the selection circuit 212, and the gating circuit 216.

[0033] The APD201 generates a charge corresponding to the incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. The cathode of the APD201 is connected to the first terminal of the quench element 202 and the input terminal of the waveform shaping unit 210. A voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD201. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD201, causing the APD201 to perform avalanche multiplication. When a charge is generated by the incident light in the APD201 with the reverse bias voltage supplied, this charge undergoes avalanche multiplication, generating an avalanche current.

[0034] When a reverse bias voltage is supplied to the APD201, there are two operating modes: Geiger mode and linear mode. Geiger mode is a mode in which the anode and cathode potential difference is greater than the breakdown voltage, while linear mode is a mode in which the anode and cathode potential difference is near or below the breakdown voltage.

[0035] An APD operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V. The APD201 may operate in linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger compared to a linear-mode APD, and the avalanche multiplication effect is more pronounced, so it is preferable to use a SPAD.

[0036] The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication. The quench element 202 suppresses the voltage supplied to the APD201, thereby suppressing avalanche multiplication (quench operation). The quench element 202 also restores the voltage supplied to the APD201 to voltage VH by flowing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 202 may be, for example, a resistor.

[0037] The waveform shaping unit 210 shapes the cathode potential change of the APD201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit can be used as the waveform shaping unit 210. Figure 5 shows an example in which one inverter is used as the waveform shaping unit 210, but the waveform shaping unit 210 may also be a circuit in which multiple inverters are connected in series, or it may be any other circuit that has a waveform shaping effect.

[0038] The gating circuit 216 is a circuit that performs gating so that the pulse signal output from the waveform shaping unit 210 passes through for a predetermined period of time. During the period in which the pulse signal can pass through the gating circuit 216, the photons incident on the APD 201 are counted by the subsequent counter circuit 211. Therefore, the gating circuit 216 controls the exposure period during which signal generation based on incident light occurs in the pixel 101. The period for which the pulse signal passes through is controlled by a control signal supplied from the vertical scanning circuit 110 via the drive line 215. Figure 5 shows an example in which one AND circuit is used as the gating circuit 216. The pulse signal and the control signal are input to the two input terminals of the AND circuit. The AND circuit outputs the logical AND of these to the counter circuit 211. Note that the gating circuit 216 can be any circuit that realizes gating, and may have a circuit configuration other than an AND circuit. Also, the waveform shaping unit 210 and the gating circuit 216 may be integrated by using a logic circuit such as a NAND circuit.

[0039] The counter circuit 211 counts the pulse signals output from the waveform shaping unit 210 via the gating circuit 216 and holds a digital signal indicating the count value. When a control signal is supplied from the vertical scanning circuit 110 via the drive line 213, the counter circuit 211 resets the signal it is holding.

[0040] The selection circuit 212 receives a control signal from the vertical scanning circuit 110 shown in Figure 4 via the drive line 214 shown in Figure 5. In response to this control signal, the selection circuit 212 switches between electrically connecting and disconnecting the counter circuit 211 and the pixel output signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to the value held in the counter circuit 211.

[0041] In the example shown in Figure 5, the selection circuit 212 switches between the electrical connection and disconnection of the counter circuit 211 and the pixel output signal line 113. However, the method for controlling the signal output to the pixel output signal line 113 is not limited to this. For example, switches such as transistors may be placed at nodes such as between the quench element 202 and the APD 201, or between the photoelectric conversion unit 102 and the pixel signal processing unit 103, to switch between electrical connection and disconnection and control the signal output to the pixel output signal line 113. Alternatively, the signal output to the pixel output signal line 113 may be controlled by changing the value of the voltage VH or voltage VL supplied to the photoelectric conversion unit 102 using a switch such as a transistor.

[0042] Figures 6(a), 6(b), and 6(c) illustrate the operation of the APD201 according to this embodiment. Figure 6(a) is a diagram showing the APD201, quench element 202, and waveform shaping unit 210 extracted from Figure 5. As shown in Figure 6(a), the connection node of the input terminals of the APD201, quench element 202, and waveform shaping unit 210 is designated as nodeA. Also, as shown in Figure 6(a), the output side of the waveform shaping unit 210 is designated as nodeB.

[0043] Figure 6(b) is a graph showing the time evolution of the potential of node A in Figure 6(a). Figure 6(c) is a graph showing the time evolution of the potential of node B in Figure 6(a). During the period from time t0 to time t1, a voltage of VH-VL is applied to APD201 in Figure 6(a). When a photon is incident on APD201 at time t1, avalanche multiplication occurs in APD201. This causes an avalanche current to flow in the quench element 202, and the potential of node A drops. Subsequently, the amount of potential drop increases further, and the voltage applied to APD201 gradually decreases. Then, at time t2, avalanche multiplication in APD201 stops. As a result, the voltage level of node A no longer drops below a certain value. Subsequently, during the period from time t2 to time t3, a current flows from the node with voltage VH to node A to compensate for the voltage drop, and at time t3, node A settles back to its original potential.

[0044] In the process described above, the potential of nodeB becomes high during the period when the potential of nodeA is below a certain threshold. In this way, the waveform of the potential drop at nodeA caused by the photon incidence is shaped by the waveform shaping unit 210 and output as a pulse to nodeB.

[0045] Next, the overall configuration and operation of the distance image generation device 30 will be described in more detail. Figure 7 is a functional block diagram showing a schematic configuration example of the distance image generation device 30 according to this embodiment. Figure 7 shows a more detailed configuration of the light-emitting device 31, light-receiving device 32, and signal processing circuit 33 described in Figure 1.

[0046] The light-emitting device 31 includes a pulse light source 311 and a light source control unit 312. The pulse light source 311 is a light source such as a semiconductor laser device that emits pulsed light throughout the entire distance measurement range. The pulse light source 311 may be a surface light source such as a surface-emitting laser. The light source control unit 312 is a control circuit that controls the light emission timing of the pulse light source 311.

[0047] The light receiving device 32 includes an imaging unit 321, a gate pulse generation unit 322, a microframe reading unit 323, a microframe addition unit 324, an addition count control unit 325, and a subframe output unit 326. As described above, the imaging unit 321 may be a photoelectric conversion device including a pixel array in which pixel circuits including the APD201 are arranged in two dimensions. This allows the distance image generation device 30 to acquire a two-dimensional distance image.

[0048] The gate pulse generation unit 322 is a control circuit that outputs a control signal to control the driving timing of the imaging unit 321. The gate pulse generation unit 322 also synchronizes the pulse light source 311 and the imaging unit 321 by sending and receiving control signals with the light source control unit 312. This makes it possible to perform imaging with controlled time differences from the time when light is emitted from the pulse light source 311 to the time when it is received by the imaging unit 321. In this embodiment, the gate pulse generation unit 322 drives the imaging unit 321 using a global gate. Global gate driving is a driving method in which several pixels (pixel groups) in the imaging unit 321 simultaneously detect incident light during the same exposure period, based on the emission time of pulsed light from the pulse light source 311. In the global gate driving of this embodiment, the detection of incident light is repeated while the collective exposure timing is sequentially shifted. As a result, each pixel of the imaging unit 321 simultaneously generates a 1-bit signal indicating the presence or absence of an incident photon during each of the multiple exposure periods.

[0049] This global gate drive is achieved by inputting a high-level signal to the input terminals of the gating circuits 216 of multiple pixels 101 for the duration of the gating period, based on a control signal from the gate pulse generation unit 322. In the processing described later, one group of multiple pixels 101 detects incident light during the same exposure period, while another group of multiple pixels 101 detects incident light during a different exposure period; however, this type of drive method is also included in global gate drive.

[0050] The microframe readout unit 323, the microframe adder unit 324, and the adder count control unit 325 are signal processing circuits that read out 1-bit signals constituting microframes from the imaging unit 321 and perform predetermined signal processing. Details of the operation of each of these units will be described later with reference to Figure 9. The subframe output unit 326 is an interface that outputs signals from the photodetector 32 to the signal processing circuit 33 in a predetermined format. The subframe output unit 326 transmits signals from the memory in the photodetector 32 to the memory in the signal processing circuit 33. The functions of each of these units can be realized by the counter circuit 211, selection circuit 212, gating circuit 216 in Figure 5, and the readout circuit 112 and output circuit 114 in Figure 4, etc.

[0051] The signal processing circuit 33 includes a subframe group storage unit 331 and a distance image generation unit 332. The signal processing circuit 33 is a computer that includes a processor operating as the distance image generation unit 332 and memory operating as the subframe group storage unit 331. The operation of each of these parts will be described later with reference to Figure 9.

[0052] Next, prior to describing the driving method of this embodiment, the configuration of the distance measurement frame, subframe, and microframe will be described with reference to Figure 8. Figure 8 schematically shows the acquisition period of the distance measurement frame, which corresponds to the distance image, the subframe used to generate the distance measurement frame, and the microframe used to generate the subframe, by arranging blocks horizontally. The horizontal direction in Figure 8 represents the passage of time, and each block represents the acquisition period of one distance measurement frame, subframe, or microframe.

[0053] A distance measurement frame F1 corresponds to a single distance image. That is, for each of the multiple pixels in a distance measurement frame F1, information corresponding to the distance to the object X is calculated from the time difference between when light is emitted and when it is received. In this embodiment, it is assumed that the distance image will be acquired as a video, and the acquisition of one distance measurement frame F1 is repeated each time a distance measurement frame period T1 has elapsed.

[0054] A single distance measurement frame F1 is generated from multiple subframes F2. A single distance measurement frame period T1 includes multiple subframe periods T2. After each subframe period T2 has elapsed, one subframe F2 is acquired repeatedly. Each subframe F2 consists of a multi-bit signal corresponding to the amount of light incident during the subframe period T2.

[0055] A single subframe F2 is generated from multiple microframes F3. A single subframe period T2 includes multiple microframe periods T3. After each microframe period T3 has elapsed, one microframe F3 is acquired repeatedly. Each microframe F3 consists of a 1-bit signal indicating the presence or absence of incident light to the photoelectric conversion element during the microframe period T3. By adding and combining multiple 1-bit signal microframes, a multi-bit signal subframe F2 is generated. Thus, a single subframe F2 may contain multi-bit signals corresponding to the number of microframes in which incident light was detected during the subframe period T2.

[0056] In this way, multiple subframes F2 are acquired, each with a different period for acquiring incident light. This signal acquisition time can be associated with the distance from the distance image generation device to the object to be measured. Then, from the distribution of signal acquisition time and signal values ​​in the multiple subframes F2, the signal acquisition time at which the signal value is maximum can be determined. Since it is estimated that reflected light was incident on the imaging unit 321 at the time of the maximum signal value, the distance can be calculated by converting the signal acquisition time at which the signal value is maximum into the distance to the object X. Furthermore, a distance image can be generated by calculating the distance for each pixel and obtaining a two-dimensional distribution of distances.

[0057] As shown in Figure 8, the length of the ranging frame period T1 required to acquire one ranging frame F1 depends on the number of subframes F2. Since the number of subframes F2 is a parameter corresponding to the number of ranging points, there is a trade-off relationship between distance resolution and frame rate.

[0058] Figure 9 shows the driving method of the distance image generation device during one distance measurement frame period T1. The driving method of this embodiment will be explained following the flowchart in Figure 9.

[0059] In the flowchart shown in Figure 9, the processing from "Start" to "End" represents the processing performed during the distance measurement frame period T1, in which one distance measurement frame F1 is acquired in Figure 8. The processing for one loop from step S11 to step S15 represents the processing performed during the subframe period T2, in which one subframe F2 is acquired in Figure 8. The processing for one loop from step S11 to step S13 represents the processing performed during the microframe period T3, in which one microframe F3 is acquired in Figure 8.

[0060] In step S11, the light source control unit 312 controls the pulse light source 311 to emit pulsed light within a predetermined distance measurement range. Synchronized with this, the gate pulse generation unit 322 controls the imaging unit 321 to start detecting incident light by global gate driving.

[0061] In step S12, the microframe reading unit 323 reads a microframe from the imaging unit 321 each time a microframe period has elapsed. The read microframes are stored in the memory of the microframe summing unit 324. This memory has a storage capacity that can hold multiple bits of data for each pixel. The microframe summing unit 324 sequentially adds the value of the microframe to the value stored in the memory each time a microframe is read. In this way, the microframe summing unit 324 adds multiple microframes within a subframe period to generate a subframe. The number of additions in the microframe summing unit 324 is controlled by the addition count control unit 325. For this control, the addition count control unit 325 holds information on the number of additions that has been set in advance. Thus, the microframe reading unit 323 functions as an acquisition unit that acquires microframes composed of 1-bit signals based on incident light to the photoelectric conversion element. The microframe summing unit 324 also functions as a synthesis unit that synthesizes multiple microframes acquired at different time periods. For example, if the number of additions is 64, a subframe signal with 6 bits of grayscale can be generated by combining 64 microframes.

[0062] In step S13, the microframe summing unit 324 determines whether the addition of a predetermined number of microframes has been completed. If the addition of the set number of microframes has not been completed (NO in step S13), the process proceeds to step S11, and the next microframe is read. If the addition of the set number of microframes has been completed (YES in step S13), the process proceeds to step S14.

[0063] In step S14, the subframe output unit 326 reads the subframes whose addition is complete from the memory of the microframe adder 324 and outputs them to the subframe group storage unit 331. The subframe group storage unit 331 stores the subframes output from the subframe output unit 326. The subframe group storage unit 331 is configured to store multiple subframes used to generate a single distance measurement frame individually for each subframe period.

[0064] In step S15, the signal processing circuit 33 determines whether the subframe group storage unit 331 has completed acquiring a predetermined number of subframes (i.e., the number of distance measurement points). If the acquisition of subframes equal to the number of distance measurement points has not been completed (NO in step S15), the process moves to step S11, where multiple microframes are acquired and added again for the reading of the next subframe. In this case, the start time of the global gate drive relative to the emission time is shifted (gate shift) by the duration of one subframe, and the same process is performed. If the acquisition of subframes equal to the number of distance measurement points has been completed (YES in step S15), the process moves to step S16. Through the loop between steps S11 and S15, subframes equal to the number of distance measurement points are acquired.

[0065] In step S16, the distance image generation unit 332 acquires multiple subframes from the subframe group storage unit 331 during one distance measurement frame period. The distance image generation unit 332 generates a distance image showing a two-dimensional distribution of distances by calculating the distance corresponding to the subframe in which the signal value is maximum for each pixel. The distance image generation unit 332 then outputs the distance image to an external device of the signal processing circuit 33. This distance image can be used, for example, to detect the surrounding environment of a vehicle. The distance image generation unit 332 may store the distance image in the internal memory of the distance image generation device.

[0066] In this embodiment, multiple pixels within the imaging unit 321 are divided into four pixel groups. The exposure period for reading out the microframe (corresponding to step S11 in Figure 9) differs for each pixel group. A specific example will be explained with reference to Figures 10(a) and 10(b). Figure 10(a) is a schematic diagram showing the arrangement of pixel groups in the pixel array according to this embodiment, and Figure 10(b) is a drive timing diagram showing the timing of the gate pulse according to this embodiment.

[0067] The pixel array of this embodiment includes a first pixel group 327A (labeled "A" in Figure 10(a)), a second pixel group 327B (labeled "B" in Figure 10(a)), a third pixel group 327C (labeled "C" in Figure 10(a)), and a fourth pixel group 327D (labeled "D" in Figure 10(a)). As shown in Figure 10(a), the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D form a two-dimensional repeating array with 4 pixels as one block.

[0068] In Figure 10(b), "Emission" indicates the emission timing of the pulse light source 311. As shown in Figure 10(b), the pulse light source 311 emits light at a constant period in accordance with the control of the light source control unit 312. This period corresponds to the length of one microframe period in which one microframe is acquired. G_A " to "P G_D " indicates the input timing of multiple types of gate pulses input from the gate pulse generation unit 322 to the imaging unit 321. G_A "P G_B "P G_C " and "P G_DThe symbols 'G01', 'G02', 'G03', and 'G04' represent different gate pulses for controlling the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D, respectively. The gate pulses G01, G02, G03, and G04 are synchronized with the light emission timing L01 and correspond to the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D, respectively. The gate pulses G01, G02, G03, and G04 reach a high level after a predetermined time has elapsed since the light emission of the pulse light source 311, but the periods during which they reach a high level within a single microframe period are different. In other words, the gate pulses G01, G02, G03, and G04 reach a high level at the 1st, 2nd, 3rd, and 4th hours, respectively, which are different from the 1st, 2nd, 3rd, and 4th hours, elapsed since the light emission of the pulse light source 311. The periods from the 1st to the 4th hours are all shorter than the length of the microframe period.

[0069] In this way, by varying the timing of the gate pulses G01, G02, G03, and G04, the exposure time for each of the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D can be varied. Therefore, four different distance measurement points can be measured within a single microframe period.

[0070] As described above, a single subframe is generated by adding multiple microframes together. That is, the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D each output signals for generating subframes with different focusing points. When a subframe is generated, at the start of the next subframe period, the timing of the gate pulse in the first pixel group 327A is gate-shifted by a predetermined time interval from the timing of the gate pulse in the current subframe period. Similarly, gate shifts are performed for the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D. Through these gate shifts, a predetermined period corresponding to each subframe period can be set as the exposure period.

[0071] In this way, by varying the exposure time for each of the multiple pixel groups and performing a gate shift for each of the multiple pixel groups for each subframe period, the number of gate shifts required to measure the required number of distance measurement points is reduced. As a result, the total period of the multiple subframes, i.e., the length of the frame period (T1 in Figure 8), is shortened. Therefore, the frame rate can be improved without reducing the number of distance measurement points (distance resolution). As described above, this embodiment provides a photoelectric converter that improves the frame rate while ensuring appropriate distance resolution.

[0072] In this embodiment, the number of pixel group types is set to four, but the number of pixel group types only needs to be at least two, and the effect of improving the frame rate can be obtained even with a number other than four. Furthermore, the number of photoelectric conversion elements included in one pixel group only needs to be at least one. That is, for two photoelectric conversion elements (first photoelectric conversion element and second photoelectric conversion element), the above effect can be obtained if the first exposure period of the first photoelectric conversion element and the second exposure period of the second photoelectric conversion element are different from each other during one microframe period. In addition, the arrangement of the pixel groups is not limited to that shown in Figure 10(a) and can be changed as appropriate.

[0073] In this embodiment, while arranging multiple pixel groups can improve the frame rate, it may also reduce the in-plane resolution of the distance measurement image. Therefore, a method may be further applied to reduce this impact on in-plane resolution by interpolating pixels with missing information in the distance measurement image using surrounding pixels.

[0074] [Second Embodiment] This embodiment describes another example of the pixel array arrangement and gate pulse timing described in the first embodiment. Elements common to the first embodiment may be omitted or simplified as appropriate.

[0075] Figure 11(a) is a schematic diagram showing the arrangement of pixel groups in the pixel array according to this embodiment, and Figure 11(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0076] The pixel array of this embodiment includes a first pixel group 328A (labeled "A" in Figure 11(a)), a second pixel group 328B (labeled "B" in Figure 11(a)), a third pixel group 328C (labeled "C" in Figure 11(a)), and a fourth pixel group 328D (labeled "D" in Figure 11(a)). The first pixel group 328A and the second pixel group 328B (first photoelectric conversion elements) are configured to be sensitive to light of a first wavelength. The third pixel group 328C and the fourth pixel group 328D (second photoelectric conversion elements) are configured to be sensitive to light of a second wavelength different from the first wavelength. More specifically, the first pixel group 328A and the second pixel group 328B are equipped with a first color filter that transmits light of the first wavelength, and the third pixel group 328C and the fourth pixel group 328D are equipped with a second color filter that transmits light of the second wavelength. In this embodiment, the ranges of the first and second wavelengths may both be, for example, in the infrared region.

[0077] The pulse light source 311 of this embodiment is configured to emit light of a first wavelength and light of a second wavelength individually at different periods. In Figure 11(b), "Emission (First Wavelength)" indicates the emission timing of light of the first wavelength by the pulse light source 311. In Figure 11(b), "Emission (Second Wavelength)" indicates the emission timing of light of the second wavelength by the pulse light source 311. As shown in Figure 11(b), the emission periods of the first wavelength and the second wavelength of light are different from each other. In the example in Figure 11(b), the emission period of the second wavelength of light is twice the emission period of the first wavelength of light.

[0078] For the first pixel group 328A and the second pixel group 328B, gate pulses G05 and G06 are input at a timing synchronized with the emission timing L02 of the first wavelength of light. For the third pixel group 328C and the fourth pixel group 328D, gate pulses G07 and G08 are input at a timing delayed by the duration of one cycle of the emission of the first wavelength of light from the emission timing L03 of the second wavelength of light. Therefore, the microframe acquisition period for the first pixel group 328A and the second pixel group 328B is different from the microframe acquisition period for the third pixel group 328C and the fourth pixel group 328D. By setting it in this way, the first pixel group 328A and the second pixel group 328B are used as pixel groups for measuring short distances (first distance range), and the third pixel group 328C and the fourth pixel group 328D are used as pixel groups for measuring long distances (second distance range). This makes it possible to measure multiple different distance ranges within the same subframe period. Furthermore, while methods that repeatedly acquire and add microframes of a 1-bit signal generally have their distance measurement range limited by the repetition period of the light emission pulse, the method of this embodiment can acquire signals from both short and long distances, enabling distance measurement over a wide range.

[0079] As described above, this embodiment provides a photoelectric converter that can obtain the same effects as the first embodiment, and can acquire multiple different distance measurement ranges simultaneously without reducing the frame rate.

[0080] In this embodiment, the bit depth of the signal of the subframe obtained from the pixel group for short-range distance measurement and the bit depth of the signal of the subframe obtained from the pixel group for long-range distance measurement may differ. If the number of summations of the signal for short-range distance measurement is 64, a short-range subframe with a bit depth of 6 bits is obtained. In this case, since the number of times the second wavelength of light is emitted is half the number of times the first wavelength of light is emitted, the number of summations of the signal for long-range distance measurement is at most 32. Therefore, the long-range subframe has a bit depth of 5 bits. In this way, if the bit depths of the two signals differ, the bit depths may be adjusted.

[0081] In this embodiment, the pulse light source 311 emits two types of wavelengths of light, but there may be three or more. Furthermore, the ratio of periods of light with different wavelengths is not limited to 2, but can be set as appropriate.

[0082] [Third Embodiment] This embodiment describes yet another example of the pixel arrangement and gate pulse timing described in the first and second embodiments. Elements common to the first or second embodiment may be omitted or simplified as appropriate.

[0083] Figure 12(a) is a schematic diagram showing the arrangement of pixel groups in the pixel array according to this embodiment, and Figure 12(b) is a drive timing diagram showing the timing of gate pulses according to this embodiment.

[0084] The pixel array of this embodiment includes a first pixel group 329R (labeled "R" in Figure 12(a)), a second pixel group 329G (labeled "G" in Figure 12(a)), a third pixel group 329B (labeled "B" in Figure 12(a)), and a fourth pixel group 329Z (labeled "Z" in Figure 12(a)). The first pixel group 329R, the second pixel group 329G, the third pixel group 329B, and the fourth pixel group 329Z are configured to be sensitive to light of different wavelengths. More specifically, the first pixel group 329R (first photoelectric conversion element) is equipped with a first color filter that transmits red light with a wavelength of 700 nm (first wavelength). The second pixel group 329G (third photoelectric conversion element) is equipped with a second color filter that transmits green light with a wavelength of 550 nm (third wavelength). The third pixel group 329B (fourth photoelectric conversion element) is equipped with a third color filter that transmits blue light with a wavelength of 430 nm (fourth wavelength). The fourth pixel group 329Z (second photoelectric conversion element) is equipped with a fourth color filter that transmits infrared light with a wavelength of 1000 nm (second wavelength).

[0085] The pulse light source 311 of this embodiment is configured to emit light of infrared wavelengths. In Figure 12(b), "Emission (Infrared Wavelength)" indicates the emission timing of infrared wavelength light by the pulse light source 311.G_Z " indicates the gate pulse for controlling the fourth pixel group 329Z. G_RGB The symbol '' indicates a control gate pulse common to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B.

[0086] For the fourth pixel group 329Z, a gate pulse G09 is input at a timing synchronized with the emission timing L04 of infrared wavelength light. For the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B, a gate pulse G10 is input over a predetermined exposure period during imaging.

[0087] As a result, signals containing information for red, green, and blue are obtained in the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B, and a color image can be generated using these signals. Note that red, green, and blue are examples, and a color image can be generated if the acquired signal contains information for multiple colors in the visible region. Also, as in the first and second embodiments, distance information is obtained in the fourth pixel group 329Z by inputting a gate pulse G09 at a timing synchronized with the emission.

[0088] As described above, according to the configuration of this embodiment, a signal for distance measurement and a signal for generating a color image can be acquired within the same subframe period. Therefore, a photoelectric converter is provided that can acquire a color image and a distance image while maintaining distance resolution and frame rate.

[0089] [Fourth Embodiment] In this embodiment, an example of a configuration in the third embodiment in which a recharge pulse for resetting the photoelectric conversion element can be input to the pixel circuit is described. Elements common to the first to third embodiments may be omitted or simplified as appropriate.

[0090] FIG. 13 is a schematic block diagram of a pixel according to the present embodiment. In FIG. 13, a transistor 217 is arranged as a specific example of the quenching element 202 in FIG. 5. The transistor 217 is an NMOS transistor. The source of the transistor 217 is connected to a connection node between the APD 201 and the input terminal of the waveform shaping unit 210. A voltage VH is supplied to the drain of the transistor 217. A recharge pulse is input to the gate of the transistor 217 from the vertical scanning circuit 110 via the drive line 218. When a high-level recharge pulse is input to the transistor 217, the transistor 217 is turned on, the cathode potential of the APD 201 is reset, and a potential enabling avalanche multiplication is obtained. This operation is called a recharge operation.

[0091] [[ID=##4]]FIG. 14 is a drive timing diagram showing the timings of the gate pulse and the recharge pulse according to the present embodiment. It is assumed that the configuration of the pixel group in the pixel array of the present embodiment is the same as that in the third embodiment. Also, the pulse light source 311 of the present embodiment is configured to be able to emit light having an infrared wavelength, similarly to the third embodiment. "P" indicates a recharge pulse for controlling the fourth pixel group 329Z. "P" indicates a recharge pulse for control common to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B. R_Z ” indicates a recharge pulse for controlling the fourth pixel group 329Z. “P R_RGB ” indicates a recharge pulse for control common to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B.

[0092] The recharge pulse synchronized with the light emission timing L05 corresponding to the fourth pixel group 329Z is defined as a recharge pulse R01. The recharge pulse synchronized with the light emission timing L05 corresponding to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B is defined as a recharge pulse R02. The recharge pulses R01 and R02 become high level after a predetermined time has elapsed since the light emission of the pulse light source 311, but the periods during which they become high level within one microframe period are different from each other.

[0093] After the recharge pulse R01 falls, the gate pulse G11 corresponding to the fourth pixel group 329Z becomes high level. Also, after the recharge pulse R02 falls, the gate pulse G12 corresponding to the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B become high level.

[0094] In the first pixel group 329R, the second pixel group 329G, and the third pixel group 329B, the period from the falling edge of the recharge pulse R02 to the falling edge of the gate pulse G12 is the photon detection period for capturing a color image. That is, in the configuration of this embodiment, the charge accumulation time for capturing a color image can be controlled by appropriately setting the timing of the recharge pulse R02 and the gate pulse G12. In the configuration of the third embodiment, since the gate pulse G10 for capturing a color image is maintained at a high level, when the subject is moving, image blur or mixing of signals from different subjects may occur. However, in the configuration of this embodiment, the effects of these factors can be reduced by appropriately setting the charge accumulation time for capturing a color image. Therefore, according to this embodiment, in addition to obtaining the same effects as in the third embodiment, image quality can be further improved.

[0095] [Fifth Embodiment] In this embodiment, an example of a configuration in which an ambient light map can be generated from microframes of multiple pixel groups is described, based on the configuration of the first to fourth embodiments. Elements common to the first to fourth embodiments may be omitted or simplified as appropriate.

[0096] Figure 15 is a functional block diagram showing a schematic configuration example of the distance measuring device according to this embodiment. In Figure 15, in addition to the configuration in Figure 7, a flag calculation unit 327 is further provided in the light receiving device 32, and a map generation unit 333 is further provided in the signal processing circuit 33. The other configurations are the same as in any of the first to fourth embodiments. The operation of the flag calculation unit 327 and the map generation unit 333 of this embodiment will be described below, assuming the configuration of the first embodiment.

[0097] The flag calculation unit 327 performs logical operations, such as logical AND, on the 1-bit signals for microframes output from multiple pixel groups, and outputs the result as an ambient light flag. The ambient light flag is a 1-bit signal indicating the presence or absence of ambient light (external light) emitted from sources other than the light-emitting device 31. For example, the flag calculation unit 327 calculates the logical AND of the four 1-bit signals output from the first pixel group 327A, the second pixel group 327B, the third pixel group 327C, and the fourth pixel group 327D. In this case, the logical value of the calculation result will be "1" if the logical values ​​of all four 1-bit signals are "1". The pixel groups to be added can be selected from multiple adjacent pixels, such as the four pixels labeled in Figure 10.

[0098] The flag calculation unit 327 generates multiple ambient light flags by performing similar logical operations on each of several regions selected from the entire pixel array. The map generation unit 333 obtains the multiple ambient light flags generated by the flag calculation unit 327, generates an ambient light map in which the logical values ​​of the ambient light flags are associated with the positions of the pixel array, and outputs it to the outside.

[0099] Figure 16 is a schematic diagram showing an example of an ambient light map according to this embodiment. Figure 16 is an image captured by a photoelectric converter mounted on a vehicle, with boxes marked "1" superimposed only on areas where the logical value of the ambient light flag is "1". As shown in Figure 16, the logical value of the ambient light flag is "1" in areas of the blue sky, where the brightness is generally high due to the influence of sunlight.

[0100] In general, in methods that measure distance by emitting light from a light source and detecting the light reflected from an object, ambient light such as sunlight, which is independent of the light source, can affect accuracy. For example, ambient light that is not reflected from the object may produce a false signal indicating the object.

[0101] Detection values ​​based on ambient light are often location-dependent. For example, the area receiving ambient light from the blue sky, as shown in Figure 16, extends over a wide area at the top of the image. In this embodiment, by calculating an ambient light flag using logical operations on 1-bit signals output from multiple pixels, it is possible to detect pixels that are likely to be incident with ambient light exhibiting a certain tendency as described above. By generating an ambient light map using this ambient light flag, information indicating areas that are likely to be incident with ambient light can be output externally. The ambient light map can be used in subsequent image processing. For example, when performing object recognition processing in an image, by excluding areas with a logical value of "1" in the ambient light map from processing, it becomes unnecessary to spend processing resources on areas with low accuracy due to ambient light, thus speeding up processing. This object recognition can be applied, for example, to autonomous driving.

[0102] As described above, this embodiment provides a photoelectric converter that can output information indicating the accuracy of a signal to the outside.

[0103] In this embodiment, a logical AND operation is given as an example of a logical operation performed in the flag calculation unit 327, but it is not limited to this; any process that outputs a value based on signals from multiple pixels is acceptable. For example, the logical operation may be a logical OR operation.

[0104] [Sixth Embodiment] Figures 17(a) and 17(b) are block diagrams of the equipment relating to the vehicle-mounted distance measuring device in this embodiment. The device 80 includes a distance measuring unit 803, which is an example of the distance image generation device of the embodiment described above, and a signal processing device (processing device) that processes signals from the distance measuring unit 803. The device 80 includes a distance measuring unit 803 that measures the distance to an object, and a collision determination unit 804 that determines whether or not there is a possibility of collision based on the measured distance. Here, the distance measuring unit 803 is an example of distance information acquisition means that acquires distance information to an object. That is, distance information is information relating to the distance to an object, etc. The collision determination unit 804 may use the distance information to determine the possibility of collision.

[0105] Device 80 is connected to a vehicle information acquisition device 810 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Device 80 is also connected to a control ECU 820, which is a control device that outputs a control signal to generate braking force on the vehicle based on the collision determination result of the collision determination unit 804. Furthermore, device 80 is connected to a warning device 830 that issues a warning to the driver based on the collision determination result of the collision determination unit 804. For example, if the collision determination result of the collision determination unit 804 indicates a high probability of collision, the control ECU 820 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 830 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seatbelt or steering wheel. These devices of device 80 function as a mobile control unit that controls the vehicle's operations as described above.

[0106] In this embodiment, the device 80 measures the distance around the vehicle, for example, in front of or behind it. Figure 17(b) shows the device when measuring the distance in front of the vehicle (distance measurement range 850). The vehicle information acquisition device 810, acting as a distance measurement control means, sends an instruction to the device 80 or the distance measurement unit 803 to perform the distance measurement operation. This configuration can further improve the accuracy of distance measurement.

[0107] The above example described controlling a vehicle to avoid collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles, or control systems that automatically stay within their lane. Furthermore, the equipment is not limited to vehicles such as automobiles, but can be applied to mobile objects (mobile devices) such as ships, aircraft, satellites, industrial robots, and consumer robots. In addition, it can be applied not only to mobile objects, but also to a wide range of devices that utilize object recognition or biometric recognition, such as intelligent transportation systems (ITS) and surveillance systems.

[0108] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways. For example, an example in which a part of the configuration of one embodiment is added to another embodiment, or an example in which a part of the configuration of one embodiment is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.

[0109] The disclosures in this specification include the complements of the concepts described herein. That is, if this specification contains a statement such as "A is B" (A=B), the specification shall be deemed to disclose or imply "A is not B" (A≠B) even if a statement such as "A is not B" is omitted. This is because the statement "A is B" presupposes that the case where "A is not B" is being considered.

[0110] The disclosures in this specification include the following components: (Composition 1) Multiple photoelectric conversion elements, An acquisition unit that acquires a microframe composed of a 1-bit signal based on the incident light on each of the plurality of photoelectric conversion elements, A synthesis unit that generates a subframe composed of multiple bit signals by combining multiple microframes, It has, The plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element, During the acquisition period of one microframe, the first exposure period in which the 1-bit signal is generated based on the incident light to the first photoelectric conversion element is performed, and the second exposure period in which the 1-bit signal is generated based on the incident light to the second photoelectric conversion element is performed, are different from each other. A photoelectric conversion device characterized by the following features. (Configuration 2) The first photoelectric conversion element is sensitive to light of a first wavelength. The second photoelectric conversion element is sensitive to light of a second wavelength that is different from the first wavelength. A photoelectric conversion device according to configuration 1, characterized in that it is a photoelectric conversion device. (Composition 3) The acquisition period of the microframe based on the incident light to the first photoelectric conversion element and the acquisition period of the microframe based on the incident light to the second photoelectric conversion element are different from each other. The photoelectric conversion device according to configuration 2, characterized in that it is a photoelectric conversion device. (Composition 4) The signal based on the incident light to the first photoelectric conversion element is used for measuring distance within a first distance range. The signal based on the incident light to the second photoelectric conversion element is used for measuring distances in a second distance range different from the first distance range. A photoelectric conversion device according to configuration 2 or 3, characterized by the above. (Composition 5) The signal based on the incident light to the first photoelectric conversion element is used to generate an image. The signal based on the incident light to the second photoelectric conversion element is used for distance measurement. The photoelectric conversion device according to configuration 2, characterized in that it is a photoelectric conversion device. (Composition 6) The plurality of photoelectric conversion elements further include a third photoelectric conversion element and a fourth photoelectric conversion element, The third photoelectric conversion element is sensitive to light of a third wavelength that is different from both the first and second wavelengths. The fourth photoelectric conversion element is sensitive to light of a fourth wavelength that is different from any of the first, second, and third wavelengths. The photoelectric conversion device according to configuration 5, characterized in that it is a photoelectric conversion device. (Composition 7) The signals based on the incident light to the first photoelectric conversion element, the third photoelectric conversion element, and the fourth photoelectric conversion element are used to generate a color image. The photoelectric conversion device according to configuration 6, characterized by the features described above. (Composition 8) The first wavelength mentioned above is a wavelength in the visible region. The second wavelength is a wavelength in the infrared region. A photoelectric conversion device according to any one of configurations 5 to 7, characterized by the above. (Composition 9) During the acquisition period of one microframe, the time at which the first photoelectric conversion element is reset and the time at which the second photoelectric conversion element is reset are different from each other. A photoelectric conversion device according to any one of configurations 1 to 8, characterized by the above. (Composition 10) The system further includes a calculation unit that generates a signal by performing a logical operation based on the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element. A photoelectric conversion device according to any one of configurations 1 to 9, characterized by the above. (Composition 11) The signal output by the calculation unit indicates the presence or absence of ambient light. A photoelectric conversion device according to configuration 10, characterized by the above. (Composition 12) The aforementioned logical operation is the logical AND of the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element. A photoelectric conversion device according to configuration 10 or 11, characterized by the above. (Composition 13) The first exposure period begins at a time when one hour has elapsed since the light source that emits light onto the object to be measured for distance measurement. The second exposure period begins at a time when a second time interval, different from the first time interval, has elapsed from the aforementioned light emission timing. A photoelectric conversion device according to any one of configurations 1 to 12, characterized by the above. (Composition 14) The aforementioned photoelectric conversion element includes an avalanche photodiode, The aforementioned 1-bit signal indicates whether or not photons were incident on the avalanche photodiode during the period in which the microframe was acquired. A photoelectric conversion device according to any one of configurations 1 to 13, characterized by the above. (Composition 15) The combining unit generates the multi-bit signal by adding the value of the 1-bit signal each time the microframe is acquired. A photoelectric conversion device according to any one of configurations 1 to 14, characterized by the above. (Composition 16) A photoelectric conversion device according to any one of configurations 1 to 15, A distance image generation unit that generates a distance image based on the signal generated by the aforementioned photoelectric conversion device, A distance image generation device characterized by comprising the following features. (Composition 17) It is a mobile object, A photoelectric conversion device according to any one of configurations 1 to 15, A mobile body control unit that controls the mobile body based on distance information acquired by the photoelectric converter, A mobile body characterized by having the following features.

[0111] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0112] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features. [Explanation of symbols]

[0113] 321 Imaging Unit 322 Gate pulse generation unit 323 Microframe Readout Unit 324 Microframe Addition Unit

Claims

1. Multiple photoelectric conversion elements, An acquisition unit acquires a microframe composed of a 1-bit signal based on the incident light on each of the plurality of photoelectric conversion elements, A synthesis unit that generates a subframe composed of multiple bit signals by combining multiple microframes, A distance image generation unit that generates a distance measurement frame using a plurality of the aforementioned subframes, It has, The plurality of photoelectric conversion elements include a first photoelectric conversion element and a second photoelectric conversion element, During the acquisition period of one microframe, the first exposure period in which the 1-bit signal is generated based on the incident light to the first photoelectric conversion element is performed, and the second exposure period in which the 1-bit signal is generated based on the incident light to the second photoelectric conversion element is performed, are different from each other. A photoelectric conversion device characterized by the following features.

2. The first photoelectric conversion element is sensitive to light of a first wavelength. The second photoelectric conversion element is sensitive to light of a second wavelength that is different from the first wavelength. The photoelectric conversion device according to feature 1.

3. The acquisition period of the microframe based on the incident light to the first photoelectric conversion element and the acquisition period of the microframe based on the incident light to the second photoelectric conversion element are different from each other. The photoelectric conversion device according to feature 2.

4. The signal based on the incident light to the first photoelectric conversion element is used for measuring distance within a first distance range. The signal based on the incident light to the second photoelectric conversion element is used for measuring distance in a second distance range different from the first distance range. The photoelectric conversion device according to feature 2.

5. The signal based on the incident light to the first photoelectric conversion element is used to generate an image. The signal based on the incident light to the second photoelectric conversion element is used for distance measurement. The photoelectric conversion device according to feature 2.

6. The plurality of photoelectric conversion elements further include a third photoelectric conversion element and a fourth photoelectric conversion element, The third photoelectric conversion element is sensitive to light of a third wavelength that is different from both the first wavelength and the second wavelength. The fourth photoelectric conversion element is sensitive to light of a fourth wavelength that is different from any of the first, second, and third wavelengths. The photoelectric conversion device according to feature 5.

7. The signals based on the incident light to the first photoelectric conversion element, the third photoelectric conversion element, and the fourth photoelectric conversion element are used to generate a color image. The photoelectric conversion device according to feature 6.

8. The first wavelength mentioned above is a wavelength in the visible region. The second wavelength is a wavelength in the infrared region. The photoelectric conversion device according to feature 5.

9. During the acquisition period of one microframe, the time at which the first photoelectric conversion element is reset and the time at which the second photoelectric conversion element is reset are different from each other. The photoelectric conversion device according to feature 1.

10. The system further includes a calculation unit that generates a signal by performing a logical operation based on the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element. The photoelectric conversion device according to feature 1.

11. The signal output by the calculation unit indicates the presence or absence of ambient light. The photoelectric conversion device according to feature 10.

12. The aforementioned logical operation is the logical AND of the 1-bit signal based on the incident light to the first photoelectric conversion element and the 1-bit signal based on the incident light to the second photoelectric conversion element. The photoelectric conversion device according to feature 10.

13. The first exposure period begins at a time when one hour has elapsed since the light source that emits light onto the object to be measured for distance measurement began to emit light. The second exposure period begins at a time when a second time interval, different from the first time interval, has elapsed from the aforementioned light emission timing. The photoelectric conversion device according to feature 1.

14. The aforementioned photoelectric conversion element includes an avalanche photodiode, The aforementioned one-bit signal indicates whether or not photons were incident on the avalanche photodiode during the period in which the microframe was acquired. The photoelectric conversion device according to feature 1.

15. The combining unit generates the multi-bit signal by adding the value of the 1-bit signal each time the microframe is acquired. The photoelectric conversion device according to feature 1.

16. It is a mobile object, A photoelectric conversion device according to any one of claims 1 to 15, A mobile body control unit that controls the mobile body based on distance information acquired by the photoelectric converter, A mobile body characterized by having the following features.

Images