Photoelectric conversion device
The photoelectric conversion device enhances light detection accuracy by alternating pixel states and using interpolation, addressing inaccuracies in non-detection periods while reducing power consumption.
Patent Information
- Application Number
- JP2023081648
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2024-11-28
AI Technical Summary
Existing photoelectric conversion devices have insufficient accuracy in detecting incident light during periods when photons are not being detected, leading to potential inaccuracies in light detection.
A photoelectric conversion device with a plurality of pixels that alternate between detection and non-detection states within a frame period, using interpolation to calculate count values based on adjacent pixels, enhancing detection accuracy.
Improves the accuracy of detecting incident light by reducing power consumption and maintaining detection precision even with fluctuating light conditions.
Smart Images

Figure 2024165438000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a photoelectric conversion device. [Background technology]
[0002] Patent Document 1 discloses an imaging device using an avalanche photodiode. The imaging device of Patent Document 1 switches between a state in which photons are detected and a state in which photons are not detected within one exposure period. This reduces power consumption in the imaging device of Patent Document 1. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-28081 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, in a photoelectric conversion device in which there is a period in which no photons are detected as in Patent Document 1, the detection accuracy of incident light may not be sufficient.
[0005] An object of the present invention is to provide a photoelectric conversion device capable of improving the detection accuracy of incident light. [Means for solving the problem]
[0006] According to one disclosure of the present specification, there is provided a photoelectric conversion device comprising: a plurality of pixels, each including a photoelectric conversion unit and a count unit that generates a count value based on light incident on the photoelectric conversion unit; and a calculation unit that performs an interpolation process on the count value, wherein each of the plurality of pixels is controlled to either a first state in which generation of the count value based on the incident light is enabled or a second state in which generation of the count value based on the incident light is disabled, the plurality of pixels include a first pixel and a second pixel, wherein in a first subframe period within one frame period, the first pixel is controlled to the first state and the second pixel is controlled to the second state, and during a second subframe period within the one frame period, the second pixel is controlled to the first state and the first pixel is controlled to the second state, and the calculation unit calculates a second count value of the first pixel in the second subframe period by the interpolation process using a first count value generated in the second subframe period at the second pixel. Effect of the Invention
[0007] According to the present invention, a photoelectric conversion device capable of improving the detection accuracy of incident light is provided. [Brief description of the drawings]
[0008] [Figure 1] 1 is a schematic diagram showing the overall configuration of a photoelectric conversion device according to a first embodiment. [Diagram 2] 1 is a schematic block diagram showing an example of the configuration of a sensor substrate according to a first embodiment. [Diagram 3] 1 is a schematic block diagram illustrating an example of the configuration of a circuit board according to a first embodiment. [Figure 4] 2 is a schematic block diagram showing an example of the configuration of one pixel of a photoelectric conversion unit and a pixel signal processing unit according to the first embodiment. FIG. [Diagram 5] 3A to 3C are diagrams illustrating the operation of the avalanche photodiode according to the first embodiment. [Figure 6] 1 is a schematic block diagram showing an example of the configuration of two pixels according to the first embodiment. FIG. [Figure 7]FIG. 2 is a diagram showing a pixel arrangement and drive timing according to the first embodiment. [Figure 8] FIG. 4 is a diagram illustrating an output value calculation unit according to the first embodiment. [Figure 9] FIG. 11 is a diagram showing a pixel array and an output value calculation unit according to the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating a pixel array and an output value calculation unit according to a third embodiment. [Figure 11] FIG. 13 is a schematic block diagram showing an example of the configuration of a circuit board according to a fourth embodiment. [Figure 12] FIG. 13 is a schematic block diagram showing an example of the configuration of two pixels according to the fourth embodiment. [Figure 13] FIG. 13 is a diagram showing a pixel array according to a fourth embodiment. [Figure 14] FIG. 13 is a diagram showing drive timing according to the fourth embodiment. [Figure 15] FIG. 13 is a diagram showing a pixel array and an output value calculation unit according to the fourth embodiment. [Figure 16] FIG. 13 is a diagram showing an arrangement of pixel blocks according to a fifth embodiment. [Figure 17] FIG. 13 is a block diagram of a light detection system according to a sixth embodiment. [Figure 18] FIG. 13 is a block diagram of a light detection system according to a seventh embodiment. [Figure 19] FIG. 13 is a schematic diagram of an endoscopic surgery system according to an eighth embodiment. [Figure 20] FIG. 13 is a schematic diagram of a light detection system according to a ninth embodiment. [Figure 21] FIG. 13 is a schematic diagram of a moving body according to a ninth embodiment. [Figure 22] 13 is a flowchart showing the operation of the light detection system according to the ninth embodiment. [Figure 23] FIG. 23 is a diagram showing a specific example of an electronic device according to a tenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment shown below is intended to embody the technical idea of the present invention, and does not limit the present invention. The size and positional relationship of the members shown in each drawing may be exaggerated for clarity of explanation. The same elements or corresponding elements in multiple drawings are given common reference numerals, and the explanation thereof may be omitted or simplified.
[0010] In the following description, terms indicating specific directions or positions (for example, "upper", "lower", "right", "left" and other terms including these terms) are used as necessary. The use of these terms is intended to facilitate understanding of the embodiments with reference to the drawings, and the technical scope of the present invention is not limited by the meanings of these terms.
[0011] [First embodiment] FIG. 1 is a schematic diagram showing the overall configuration of a photoelectric conversion device 100 according to the present embodiment. The photoelectric conversion device 100 may be, for example, a solid-state imaging device, a focus detection device, a distance measuring device, a TOF (Time-Of-Flight) camera, or the like. The photoelectric conversion device 100 has a sensor substrate 11 and a circuit substrate 21 that are stacked on each other. The sensor substrate 11 and the circuit substrate 21 are electrically connected to each other. The sensor substrate 11 has a pixel region 12 in which a plurality of pixel circuits 101 are arranged to form a plurality of rows and a plurality of columns. The circuit substrate 21 has a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged to form a plurality of rows and a plurality of columns, and a second circuit region 23 arranged on the periphery of the first circuit region 22. The second circuit region 23 may include a circuit for controlling the plurality of pixel signal processing units 103, and the like. The sensor substrate 11 has a light incident surface that receives incident light and a connection surface that faces the light incident surface. The connection surface of the sensor substrate 11 is connected to the circuit substrate 21. In other words, the photoelectric conversion device 100 is a so-called back-illuminated type.
[0012] In this specification, "planar view" refers to a view from a direction perpendicular to the surface opposite to the light incident surface. Also, a cross section refers to a surface in a direction perpendicular to the surface opposite to the light incident surface of the sensor substrate 11. Note that the light incident surface may be rough when viewed microscopically, and in such a case, the planar view is defined based on the light incident surface when viewed macroscopically.
[0013] In the following description, the sensor substrate 11 and the circuit substrate 21 are described as being diced chips, but the sensor substrate 11 and the circuit substrate 21 are not limited to being chips. For example, the sensor substrate 11 and the circuit substrate 21 may be wafers. In addition, when the sensor substrate 11 and the circuit substrate 21 are diced chips, the photoelectric conversion device 100 may be manufactured by stacking them in a wafer state and then dicing them, or may be manufactured by stacking the chips after dicing.
[0014] FIG. 2 is a schematic block diagram showing a configuration example of the sensor substrate 11 according to the present embodiment. In the pixel region 12, a plurality of pixel circuits 101 are arranged in a plurality of rows and a plurality of columns. Each of the plurality of pixel circuits 101 has a photoelectric conversion unit 102 including an avalanche photodiode (hereinafter referred to as APD) as a photoelectric conversion element. When the photoelectric conversion device 100 is an imaging device, the plurality of pixel circuits 101 may be an element that generates a signal for an image by photoelectric conversion. However, when the photoelectric conversion device 100 is a distance measuring device using a technology such as TOF, the pixel circuit 101 may be an element for measuring the time when light arrives and the amount of light. That is, the use of the plurality of pixel circuits 101 is not limited to image acquisition.
[0015] The conductivity type of the charge pair generated in the APD and used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type in which the charge of the same polarity as the signal charge is the majority carrier. The conductivity type opposite to the first conductivity type, that is, the conductivity type in which the charge of the opposite polarity to the signal charge is the majority carrier, is called the second conductivity type. In the APD described below, the anode of the APD is at a fixed potential, and a signal is taken out from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is an N-type semiconductor region, and the semiconductor region of the second conductivity type is a P-type semiconductor region. The cathode of the APD may be at a fixed potential, and the signal may be taken out from the anode of the APD. In this case, the semiconductor region of the first conductivity type is a P-type semiconductor region, and the semiconductor region of the second conductivity type is an N-type semiconductor region. In the following, a case in which one node of the APD is at a fixed potential will be described, but the potentials of both nodes may be fluctuating.
[0016] In this specification, when the term "impurity concentration" is used simply, it means the net impurity concentration minus the amount compensated for by the impurity of the opposite conductivity type. In other words, "impurity concentration" refers to the NET doping concentration. A region where the P-type doped impurity concentration is higher than the N-type doped impurity concentration is a P-type semiconductor region. Conversely, a region where the N-type doped impurity concentration is higher than the P-type doped impurity concentration is an N-type semiconductor region.
[0017] 3 is a schematic block diagram showing an example of the configuration of a circuit board 21 according to this embodiment. The circuit board 21 has a first circuit region 22 in which a plurality of pixel signal processing units 103 are arranged in a plurality of rows and a plurality of columns.
[0018] Further, a vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, a pixel output signal line 113, an output circuit 114, and a control signal generating unit 115 are arranged on the circuit board 21. The multiple photoelectric conversion units 102 shown in Fig. 2 and the multiple pixel signal processing units 103 shown in Fig. 3 are electrically connected to each other via connection wiring provided for each pixel circuit 101.
[0019] The control signal generating unit 115 is a control circuit that generates control signals for driving the vertical scanning circuit 110, the horizontal scanning circuit 111, and the readout circuit 112, and supplies these circuits with these signals. In this way, the control signal generating unit 115 controls the drive timing of each circuit.
[0020] The vertical scanning circuit 110 supplies a control signal to each of the pixel signal processing units 103 based on the control signal supplied from the control signal generation unit 115. The vertical scanning circuit 110 supplies a control signal to each pixel signal processing unit 103 for each row via a drive line provided for each row of the first circuit area 22. As will be described later, there may be multiple drive lines for each row. The vertical scanning circuit 110 may include logic circuits such as a shift register and an address decoder. In this way, the vertical scanning circuit 110 selects a row to which a signal is to be output from the pixel signal processing unit 103.
[0021] A signal output from the photoelectric conversion unit 102 of the pixel circuit 101 is processed by a pixel signal processing unit 103. The pixel signal processing unit 103 counts the number of pulses output from the APD included in the photoelectric conversion unit 102 to obtain and hold a digital signal.
[0022] The horizontal scanning circuit 111 supplies a control signal to the readout circuit 112 based on the control signal supplied from the control signal generation unit 115. The pixel signal processing unit 103 is connected to the readout circuit 112 via a pixel output signal line 113 provided for each column of the first circuit region 22. The pixel output signal line 113 of one column is shared by a plurality of pixel signal processing units 103 of the corresponding column. The pixel output signal line 113 includes a plurality of wirings, and has at least a function of outputting a digital signal from each pixel signal processing unit 103 to the readout circuit 112 and a function of supplying a control signal for selecting a column for outputting a signal to the pixel signal processing unit 103. The readout circuit 112 outputs a signal to a storage unit or a signal processing unit outside the photoelectric conversion device 100 via the output circuit 114 based on the control signal supplied from the control signal generation unit 115.
[0023] The photoelectric conversion units 102 in the pixel region 12 may be arranged one-dimensionally. Moreover, it is not necessary to provide one pixel signal processing unit 103 for each pixel circuit 101. For example, one pixel signal processing unit 103 may be shared by a plurality of pixel circuits 101. In this case, the pixel signal processing unit 103 provides a signal processing function to each pixel circuit 101 by sequentially processing signals output from each photoelectric conversion unit 102.
[0024] 2 and 3, a first circuit region 22 including a plurality of pixel signal processing units 103 is arranged in a region overlapping the pixel region 12 in a plan view. A vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generating unit 115 are arranged so as to overlap between an end of the sensor substrate 11 and an end of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has the pixel region 12 and a non-pixel region arranged around the pixel region 12. A second circuit region 23 including the vertical scanning circuit 110, the horizontal scanning circuit 111, the readout circuit 112, the output circuit 114, and the control signal generating unit 115 is arranged in a region of the circuit substrate 21 overlapping the non-pixel region in a plan view.
[0025] The arrangement of the pixel output signal lines 113, the readout circuits 112, and the output circuits 114 are not limited to those shown in Fig. 3. For example, the pixel output signal lines 113 may be arranged to extend in the row direction and shared by a plurality of pixel signal processing units 103 in the corresponding row. The readout circuits 112 may be arranged so that the pixel output signal lines 113 in each row are connected to each other.
[0026] Fig. 4 is a schematic block diagram showing a configuration example of one pixel of the photoelectric conversion unit 102 and pixel signal processing unit 103 according to this embodiment. Fig. 4 shows a more specific configuration example including a connection relationship between the photoelectric conversion unit 102 arranged on the sensor substrate 11 and the pixel signal processing unit 103 arranged on the circuit substrate 21. In Fig. 4, the drive lines between the vertical scanning circuit 110 and the pixel signal processing unit 103 in Fig. 3 are shown as drive lines 213 and 214.
[0027] The photoelectric conversion unit 102 has an APD 201. The pixel signal processing unit 103 has a quenching element 202, a waveform shaping unit 210, a counting unit 211, and a selection circuit 212. It is sufficient that the pixel signal processing unit 103 has at least one of the waveform shaping unit 210, the counting unit 211, and the selection circuit 212.
[0028] The APD201 is a photoelectric conversion unit that generates charge pairs according to incident light by photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. A cathode of the APD201 is connected to a first terminal of the quench element 202 and an input terminal of the waveform shaping unit 210. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD201. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD201 such that the APD201 performs avalanche multiplication. When charges are generated by incident light in the APD201 to which the reverse bias voltage is supplied, the charges cause avalanche multiplication, generating an avalanche current.
[0029] In addition, there are two operation modes when a reverse bias voltage is supplied to the APD 201: Geiger mode and linear mode. The Geiger mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is greater than the breakdown voltage, and the linear mode is a mode in which the APD 201 operates with a potential difference between the anode and cathode that is close to or less than the breakdown voltage.
[0030] An APD operated in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the voltage VL (first voltage) is −30 V and the voltage VH (second voltage) is 1 V. The APD 201 may be operated in either linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger than that of an APD in linear mode, and the effect of avalanche multiplication is more pronounced, so a SPAD is preferable.
[0031] The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication. The quench element 202 suppresses the voltage supplied to the APD 201 to suppress avalanche multiplication (quench operation). The quench element 202 also returns the voltage supplied to the APD 201 to the voltage VH by passing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 202 may be, for example, a P-type MOS transistor. In this case, a potential is input to the gate of the P-type MOS transistor as a control signal, thereby controlling the P-type MOS transistor to be on or off, and controlling whether the recharge operation is enabled or disabled. As a result, the pixel is controlled to either a detection state in which a count value corresponding to incident light can be generated, or a non-detection state in which a count value is not generated.
[0032] The waveform shaping unit 210 is a circuit that shapes the potential change of the cathode of the APD 201 obtained when a photon is detected, and outputs a pulse signal. For example, an inverter circuit is used for the waveform shaping unit 210. Although an example in which one inverter is used as the waveform shaping unit 210 is shown in Fig. 4, the waveform shaping unit 210 may be a circuit in which a plurality of inverters are connected in series, or may be another circuit having a waveform shaping effect.
[0033] The count unit 211 counts the number of pulses output from the waveform shaping unit 210 and holds a digital signal indicating the count value. When a control signal is supplied from the vertical scanning circuit 110 via the drive line 213, the count unit 211 resets the signal it holds.
[0034] A control signal is supplied to the selection circuit 212 from the vertical scanning circuit 110 shown in Fig. 3 via a drive line 214 shown in Fig. 4. In response to this control signal, the selection circuit 212 switches between electrical connection and non-connection between the count unit 211 and the pixel output signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to a value held in the count unit 211.
[0035] 4, the selection circuit 212 switches between electrical connection and disconnection between the count unit 211 and the pixel output signal line 113, but the method of controlling the signal output to the pixel output signal line 113 is not limited to this. For example, a switch such as a transistor may be disposed at a node between the quench element 202 and the APD 201, between the photoelectric conversion unit 102 and the pixel signal processing unit 103, or the like, and the signal output to the pixel output signal line 113 may be controlled by switching between electrical connection and disconnection. Alternatively, the signal output to the pixel output signal line 113 may be controlled by changing the value of the voltage VH or voltage VL supplied to the photoelectric conversion unit 102 using a switch such as a transistor.
[0036] 4 shows a configuration example using a counting unit 211 including a counter. However, instead of the counting unit 211, a time to digital converter (hereinafter, TDC) and a memory may be used to acquire the timing for detecting a pulse. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. In this case, a control signal (reference signal) may be supplied to the TDC from the vertical scanning circuit 110 in FIG. 3 via a driving line. The TDC acquires a signal indicating the relative time of the input timing of the pulse based on the control signal as a digital signal.
[0037] Figures 5(a), 5(b) and 5(c) are diagrams for explaining the operation of the APD 201 according to this embodiment. Figure 5(a) is a diagram showing the APD 201, the quench element 202 and the waveform shaping unit 210 extracted from Figure 4. As shown in Figure 5(a), a connection node between the APD 201, the quench element 202 and the input terminals of the waveform shaping unit 210 is referred to as nodeA. Also, as shown in Figure 5(a), the output side of the waveform shaping unit 210 is referred to as nodeB.
[0038] FIG. 5(b) is a graph showing the time change of the potential of nodeA in FIG. 5(a). FIG. 5(c) is a graph showing the time change of the potential of nodeB in FIG. 5(a). In the period from time t0 to time t1, a voltage of VH-VL is applied to the APD 201 in FIG. 5(a). When a photon is incident on the APD 201 at time t1, avalanche multiplication occurs in the APD 201. As a result, an avalanche current flows through the quench element 202, and the potential of nodeA drops. After that, the amount of potential drop further increases, and the voltage applied to the APD 201 gradually decreases. Then, at time t2, the avalanche multiplication in the APD 201 stops. As a result, the voltage level of nodeA does not drop below a certain value. After that, in the period from time t2 to time t3, a current that compensates for the voltage drop flows from the node of voltage VH to nodeA, and at time t3, nodeA settles to its original potential.
[0039] In the above process, the potential of node B becomes high during the period when the potential of node A is lower than a certain threshold. In this way, the waveform of the drop in the potential of node A caused by the incidence of a photon is shaped by the waveform shaping unit 210 and output as a pulse to node B.
[0040] The configuration and operation of the photoelectric conversion device 100 of this embodiment will be described in more detail below with reference to FIGS.
[0041] FIG. 6 is a schematic block diagram showing an example of the configuration of two pixels of the photoelectric conversion device 100 according to the present embodiment. FIG. 6 shows a more specific configuration of two vertically adjacent pixels (the photoelectric conversion unit 102 and the pixel signal processing unit 103). As shown in FIG. 1 to FIG. 3, a plurality of pixels are arranged to form a plurality of rows and a plurality of columns, and FIG. 6 shows two pixels extracted from the plurality of pixels. These two pixels are called pixel A (first pixel) and pixel B (second pixel). Each of pixel A and pixel B has an APD 201, a quenching element 202, a waveform shaping unit 210, and a counting unit 211. The counting unit 211 has a counter 221, a memory 222, an output value calculation unit 223, and a control unit 224. Note that in FIG. 6, the selection circuit 212 shown in FIG. 4 is omitted.
[0042] The counter 221 counts the pulses output from the waveform shaping section 210. The memory 222 holds the count value output from the counter 221 based on a latch signal (not shown).
[0043] The control unit 224 receives a control signal output from the control circuit and controls the quench element 202. As a result, the control unit 224 switches between a detection state (first state) in which generation of a count value according to incident light to the APD 201 is enabled, and a non-detection state (second state) in which generation of a count value according to incident light to the APD 201 is disabled. For example, when the quench element 202 is a P-type MOS transistor, the control unit 224 realizes this state switching by controlling the potential of the gate of the P-type MOS transistor. The pixel A and the pixel B can be controlled to different states. That is, the pixel A and the pixel B can be controlled to switch between a state in which the pixel A is in a detection state and the pixel B is in a non-detection state, and a state in which the pixel A is in a non-detection state and the pixel B is in a detection state.
[0044] The output value calculation unit 223 obtains the count value held in the memory 222 for pixel A and the count value held in the memory 222 for pixel B, performs a predetermined calculation, and outputs the result to the readout circuit 112. The details of this calculation process will be described later.
[0045] Fig. 7(a) is a schematic diagram showing a pixel arrangement in the photoelectric conversion device 100 according to this embodiment, and Fig. 7(b) is a diagram showing drive timing in the photoelectric conversion device 100 according to this embodiment.
[0046] FIG. 7(a) shows five rows and five columns of a plurality of pixels arranged in a plurality of rows and a plurality of columns. "A" in FIG. 7(a) shows pixel A in FIG. 6, and "B" in FIG. 7(a) shows pixel B in FIG. 6. As shown in FIG. 7(a), either pixel A or pixel B is arranged in one row, and the row in which pixel A is arranged and the row in which pixel B is arranged are arranged alternately. That is, pixels A and pixels B are arranged in a striped pattern. A pixel block BL of two rows and one column shown in FIG. 7(a) is one unit of the arrangement of pixels A and pixels B.
[0047] FIG. 7(b) is an explanatory diagram of the drive timing of pixel A and pixel B. FIG. 7(b) shows the detection or non-detection operation, the count value of the counter 221, and the value stored in the memory 222 for each of pixel A and pixel B. FIG. 7(b) shows the operation for two frame periods. One frame period includes two sub-frame periods. Times t10, t11, t12, t13, and t14 are the start and end times of the sub-frame periods, respectively. Also, times t10, t12, and t14 are the start and end times of the frame period, respectively.
[0048] At time t10, the first subframe period begins. During the first subframe period, pixel A is in a detection state, and pixel B is in a non-detection state. In pixel A, counter 221 starts counting photons from time t10. At time t11, the first subframe period ends. The count value "150" of counter 221 of pixel A at time t11 is held in memory 222 of pixel A.
[0049] At time t11, the second subframe period begins. During the second subframe period, pixel A is in a non-detection state, and pixel B is in a detection state. In pixel B, counter 221 starts counting photons from time t11. At time t12, the second subframe period ends. The count value "200" of counter 221 of pixel B at time t12 is held in memory 222 of pixel B.
[0050] At time t12, a count value (third count value) corresponding to the incident light in the first sub-frame period is held in memory 222 of pixel A, and a count value (first count value) corresponding to the incident light in the second sub-frame period is held in memory 222 of pixel B. In this manner, the operation for one frame period from time t10 to time t12 is completed.
[0051] In pixel A, incident light is not detected in the second sub-frame period from time t11 to time t12. A signal corresponding to this period is interpolated by the output value calculation unit 223 of pixel A using at least the output signal of pixel B in the second sub-frame period. In addition, in pixel B, incident light is not detected in the first sub-frame period from time t10 to time t11. A signal corresponding to this period is interpolated by the output value calculation unit 223 of pixel B using at least the output signal of pixel A in the first sub-frame period. This interpolation process is performed in the period from time t12 to time t14, and the processed signal is read out to the readout circuit 112 via the pixel output signal line 113.
[0052] In the period from time t12 to time t14, processing for the next one frame period is carried out. The processing for this period is generally similar to the processing from time t10 to time t12, and therefore a description thereof will be omitted.
[0053] Fig. 8 is a schematic diagram showing the output value calculation unit 223 of pixel A in this embodiment. As shown in Fig. 8, the output value of the memory 222 of pixel A and the output value of the memory 222 of pixel B are input to the output value calculation unit 223 of pixel A. Based on these, the output value calculation unit 223 of pixel A performs the calculation of the following equation (1).
number
[0054] In equation (1), Aout is the output value of output value calculation unit 223 for pixel A, Ain is the output value of memory 222 for pixel A, Bin is the output value of memory 222 for pixel B, and α is an interpolation parameter. The interpolation parameter α is a parameter that determines the weighting of Ain and Bin, and is determined in advance within the range of 0<α≦1. As described above, Ain is the count value of pixel A detected in the first subframe period, and Bin is the count value of pixel B detected in the second subframe period. Note that the interpolation parameter α may be dynamically determined based on input signals such as Ain and Bin.
[0055] The first term of the formula (1) is the count value (third count value) of pixel A detected in the first subframe period. The second term of the formula (1) is an interpolated value (second count value) corresponding to the count value of pixel A in the second subframe period. As shown in the formula (1), the output value calculation unit 223 adds the first term, which is the count value of pixel A detected in the first subframe period, to the second term, which is an interpolated value, and outputs the result. As shown in the formula (1), this interpolated value is calculated based on at least the output value of the memory 222 of pixel B. When the interpolation parameter α is 1, this interpolated value is calculated based only on the output value of the memory 222 of pixel B. Also, when the interpolation parameter α is not 1, this interpolated value is calculated based on the output value of the memory 222 of pixel A and the output value of the memory 222 of pixel B.
[0056] In this embodiment, as shown in Fig. 7(b), some of the pixels are in a non-detection state during one frame period, which reduces power consumption compared to when all of the pixels are in a detection state during one frame period.
[0057] Since a pixel in a non-detection state cannot detect incident light, there are cases where sufficient detection accuracy cannot be obtained depending on the state of the incident light. Examples of such cases include cases where the incident light fluctuates significantly in a short period of time due to the movement of a subject, flash light, etc. However, in this embodiment, even if the incident light to pixel A fluctuates significantly during a subframe period in which pixel A is in a non-detection state, the detection accuracy can be maintained by performing an interpolation process using the output value of the adjacent pixel B. Therefore, according to this embodiment, a photoelectric conversion device that can improve the detection accuracy of incident light is provided.
[0058] Note that performing interpolation using output values of different pixels may result in aliasing. As shown in formula (1), when the interpolation parameter α is not 1, an interpolation is performed in which the output value of pixel A in the previous subframe period is weighted and added to the output value of pixel B. This makes it possible to perform an interpolation that takes into account the balance between the influence of aliasing and the reduction of the influence of fluctuations in the incident light.
[0059] [Second embodiment] A photoelectric conversion device 100 according to the second embodiment will be described with reference to Fig. 9(a) to Fig. 9(c). In the description of this embodiment, the description of elements common to the first embodiment may be omitted. In this embodiment, the difference from the first embodiment is that the output value calculation unit 223 of pixel A performs interpolation based on the output values of the memory 222 of a plurality of pixels B.
[0060] Fig. 9(a) is a schematic diagram showing a pixel array in the photoelectric conversion device 100 according to this embodiment. Fig. 9(b) is a schematic diagram showing the positions of multiple pixels B used in the interpolation process for the central pixel A. Fig. 9(c) is a schematic diagram showing the output value calculation unit 223 for pixel A in this embodiment.
[0061] The pixel array shown in Fig. 9(a) is generally similar to that in Fig. 7(a), and therefore a description thereof will be omitted. As shown in Fig. 9(b), two pixels B above and below pixel A are used in the interpolation process of pixel A. The pixel above pixel A is called pixel B1, and the pixel below pixel A is called pixel B2.
[0062] 9(c), the output value of the memory 222 of pixel A, the output value of the memory 222 of pixel B1, and the output value of the memory 222 of pixel B2 are input to the output value calculation unit 223 of pixel A. Based on these, the output value calculation unit 223 of pixel A performs the calculation of the following equation (2).
number
[0063] In equation (2), Aout is the output value of output value calculation unit 223 of pixel A, Ain is the output value of memory 222 of pixel A, B1in is the output value of memory 222 of pixel B1, B2in is the output value of memory 222 of pixel B2, and β is an interpolation parameter. The output value calculation unit 223 of pixel A has a parameter calculation unit 225 that calculates the interpolation parameter β. The interpolation parameter β is a parameter that determines the weighting of B1in and B2in, and is determined in the range of 0<β≦1.
[0064] The first term of equation (2) is the count value of pixel A detected in the first subframe period. The second term of equation (2) is an interpolated value corresponding to the count value of pixel A in the second subframe period. As shown in equation (2), output value calculation unit 223 adds the first term, which is the count value of pixel A detected in the first subframe period, to the second term, which is an interpolated value, and outputs the result. As shown in equation (2), this interpolated value is calculated based on at least the output value of memory 222 of pixel B2. When the interpolation parameter β is 1, this interpolated value is calculated based only on the output value of memory 222 of pixel B2. When the interpolation parameter β is not 1, this interpolated value is calculated based on the output value of memory 222 of pixel B1 and the output value of memory 222 of pixel B2.
[0065] The parameter calculation unit 225 calculates the interpolation parameter β by comparing the output value of pixel A with the output value of pixel B1 and the output value of pixel B2. For example, by comparing the output values of pixel A with pixel B1 or pixel A with pixel B2, it is possible to detect the presence of an edge portion of an object at a position corresponding to these pixels, the movement of the object, etc. Based on the results of these comparisons, the parameter calculation unit 225 calculates the interpolation parameter β.
[0066] According to the present embodiment, a photoelectric conversion device capable of improving the detection accuracy of incident light is provided, similarly to the first embodiment. Moreover, in the present embodiment, a larger number of pixels B are used in the interpolation process than in the first embodiment, so that the detection accuracy can be further improved.
[0067] [Third embodiment] A photoelectric conversion device 100 according to the third embodiment will be described with reference to Figs. 10(a) to 10(c). In the description of this embodiment, elements common to the first and second embodiments may be omitted. This embodiment is a modified example of the second embodiment. This embodiment differs from the second embodiment in that the output value calculation unit 223 of pixel A performs interpolation based on the output values of the memory 222 of four pixels B.
[0068] Fig. 10(a) is a schematic diagram showing a pixel array in a photoelectric conversion device 100 according to this embodiment. Fig. 10(b) is a schematic diagram showing the positions of multiple pixels B used in an interpolation process for a central pixel A. Fig. 10(c) is a schematic diagram showing an output value calculation unit 223 for pixel A in this embodiment.
[0069] 10(a), pixels A and B are alternately arranged in one row, and pixels A and B are also alternately arranged in one column, i.e., pixels A and B are arranged in a checkered pattern.
[0070] As shown in Fig. 9(b), four pixels B on the top, bottom, left and right of pixel A are used in the interpolation process of that pixel. The pixel above pixel A is called pixel B1, and the pixel below pixel A is called pixel B2. In addition, the pixel to the left of pixel A is called pixel B3, and the pixel to the right of pixel A is called pixel B4.
[0071] 10(c), the output value of the memory 222 for pixel A and the output values of the memory 222 for pixels B1 to B4 are input to the output value calculation unit 223 for pixel A. Based on these, the output value calculation unit 223 for pixel A performs the calculation of the following equation (3).
number
[0072] In equation (3), Aout is the output value of the output value calculation unit 223 of pixel A, Ain is the output value of the memory 222 of pixel A, B1in to B4in are the output values of the memory 222 of pixels B1 to B4, respectively, and a, b, c, and d are interpolation parameters. The output value calculation unit 223 of pixel A has a parameter calculation unit 225 that calculates the interpolation parameters a, b, c, and d. The interpolation parameters a, b, c, and d are parameters that determine the weighting of B1in to B4in, and are determined so as to satisfy a+b+c+d=1.
[0073] The first term of equation (3) is the count value of pixel A detected in the first subframe period. The second term of equation (3) is an interpolated value corresponding to the count value of pixel A in the second subframe period. As shown in equation (3), output value calculation unit 223 adds together the first term, which is the count value of pixel A detected in the first subframe period, and the second term, which is an interpolated value, and outputs the sum. This interpolated value is calculated based on the output values of memory 222 for pixels B1 to B4, as shown in equation (3).
[0074] The parameter calculation unit 225 calculates the interpolation parameters a, b, c, and d by comparing the output value of pixel A with the output values of pixels B1 to B4. For example, by comparing the output values of pixels A, B1, B2, B3, and B4, it is possible to detect the presence of an edge portion of an object at the positions corresponding to these pixels, the movement of the object, and the like. Based on the results of these comparisons, the parameter calculation unit 225 calculates the interpolation parameters a, b, c, and d.
[0075] According to the present embodiment, a photoelectric conversion device capable of improving the detection accuracy of incident light is provided, similarly to the first and second embodiments. Moreover, in the present embodiment, a larger number of pixels B are used in the interpolation process than in the second embodiment, so that the detection accuracy can be further improved.
[0076] In the second and third embodiments, the number of pixels B used in the interpolation process is two and four, respectively, but is not limited thereto. For example, the number of pixels B used in the interpolation process may be four or more.
[0077] [Fourth embodiment] A photoelectric conversion device 100 according to a fourth embodiment will be described with reference to Fig. 11 to Fig. 15(b). In the description of this embodiment, elements common to the first to third embodiments may be omitted. This embodiment differs from the first to third embodiments in that the pixel block BL has two rows and two columns, and one frame period is divided into four subframe periods.
[0078] 11 is a schematic block diagram showing an example of the configuration of the circuit board 21 according to this embodiment. An output value calculation unit 223 is arranged on the circuit board 21. The output value calculation unit 223 has a function of performing an interpolation process similar to the output value calculation unit 223 described in the first to third embodiments. The output value calculation unit 223 processes signals input via a plurality of pixel output signal lines 113 arranged corresponding to each column for each column. That is, signals output from a plurality of pixels in one column are input to the output value calculation unit 223 via a common pixel output signal line 113 and processed.
[0079] FIG. 12 is a schematic block diagram showing an example of the configuration of two pixels of the photoelectric conversion device 100 according to this embodiment. As shown in FIG. 11, in this embodiment, the output value calculation unit 223 is arranged outside the pixel. Therefore, in FIG. 12, unlike FIG. 6, the output value calculation unit 223 is not arranged inside the pixel. In this embodiment, output signals from many pixels are used for processing by the output value calculation unit 223, so the circuit scale of the output value calculation unit 223 is large. By arranging the output value calculation unit 223 outside the pixel instead of inside the pixel, the area occupied by the pixel is prevented from becoming large. Note that, as will be described later, in this embodiment, pixels C and D are arranged in the pixel block BL, but are not shown in FIG. 12.
[0080] Fig. 13 is a schematic diagram showing a pixel array in a photoelectric conversion device 100 according to this embodiment. In this embodiment, a pixel block BL has four pixels arranged in two rows and two columns. These four pixels are called pixel A (first pixel), pixel B (second pixel), pixel C (third pixel), and pixel D (fourth pixel). "A", "B", "C", and "D" in Fig. 13 respectively indicate pixel A, pixel B, pixel C, and pixel D.
[0081] FIG. 14 is an explanatory diagram of the drive timing of pixel A, pixel B, pixel C, and pixel D. FIG. 14 shows the detection or non-detection operation, the count value of the counter 221, and the value stored in the memory 222 for each of pixel A, pixel B, pixel C, and pixel D. FIG. 14 shows the operation of one frame period and one sub-frame period. One frame period includes four sub-frame periods. Times t20, t21, t22, t23, t24, and t25 are the start and end times of the sub-frame periods, respectively. Also, times t20 and t24 are the start and end times of the frame period, respectively.
[0082] At time t20, the first subframe period begins. In the first subframe period, pixel A is in a detection state, and pixels B, C, and D are in a non-detection state. In pixel A, counter 221 starts counting photons from time t10. At time t21, the first subframe period ends. The count value "150" of counter 221 of pixel A at time t21 is held in memory 222 of pixel A.
[0083] At time t21, the second subframe period begins. During the second subframe period, pixel B is in a detection state, and pixels A, C, and D are in a non-detection state. In pixel B, counter 221 starts counting photons from time t21. At time t22, the second subframe period ends. The count value "200" of counter 221 of pixel B at time t22 is held in memory 222 of pixel B.
[0084] At time t22, the third subframe period begins. In the third subframe period, pixel C is in a detection state, and pixels A, B, and D are in a non-detection state. In pixel C, counter 221 starts counting photons from time t22. At time t23, the third subframe period ends. The count value "180" of counter 221 of pixel C at time t23 is held in memory 222 of pixel C.
[0085] At time t23, the fourth subframe period begins. In the fourth subframe period, pixel D is in a detection state, and pixels A, B, and C are in a non-detection state. In pixel D, counter 221 starts counting photons from time t23. At time t24, the fourth subframe period ends. The count value "190" of counter 221 of pixel D at time t24 is held in memory 222 of pixel C.
[0086] At time t24, the memory 222 of pixel A holds a count value (third count value) corresponding to the incident light in the first sub-frame period, and the memory 222 of pixel B holds a count value (first count value) corresponding to the incident light in the second sub-frame period. Furthermore, the memory 222 of pixel C holds a count value (fourth count value) corresponding to the incident light in the third sub-frame period, and the memory 222 of pixel D holds a count value (fifth count value) corresponding to the incident light in the fourth sub-frame period. In this manner, the operation for one frame period from time t20 to time t24 is completed.
[0087] In pixel A, incident light is not detected during the second subframe period from time t21 to time t24 to the fourth subframe period. Signals corresponding to these periods are interpolated by the output value calculation unit 223 of pixel A using the output signal of the second subframe period of pixel B, the output signal of the third subframe period of pixel C, and the output signal of the fourth subframe period of pixel D. In addition, in pixel B, incident light is not detected during the first subframe period from time t20 to time t21 and the third and fourth subframe periods from time t22 to time t24. Signals corresponding to these periods are interpolated by the output value calculation unit 223 of pixel B using the output signal of the first subframe period of pixel A, the output signal of the third subframe period of pixel C, and the output signal of the fourth subframe period of pixel D. Similar interpolation processing is performed for pixels C and D. These interpolation processing is performed during the period after time t24, and the processed signals are read out to the readout circuit 112 via the pixel output signal line 113.
[0088] In the period after time t24, processing for the next one frame period is carried out. The processing for this period is similar to the processing from time t20 to time t24, and therefore a description thereof will be omitted.
[0089] Fig. 15(a) is a schematic diagram showing the positions of pixels B, C, and D used in the interpolation process for central pixel A. Fig. 15(b) is a schematic diagram showing the output value calculation unit 223 for pixel A in this embodiment.
[0090] As shown in Figure 15(a), two pixels B above and below pixel A, two pixels C to the left and right of pixel A, and four pixels D diagonally adjacent to pixel A are used in the interpolation process of pixel A. The pixel above pixel A is called pixel B1, and the pixel below pixel A is called pixel B2. The pixel to the left of pixel A is called pixel C1, and the pixel to the right of pixel A is called pixel C2. The pixel to the top left of pixel A is called pixel D1, the pixel to the top right of pixel A is called pixel D2, the pixel to the bottom left of pixel A is called pixel D3, and the pixel to the bottom right of pixel A is called pixel D4.
[0091] 15(c), the output values of pixels A, B1, B2, C1, C2, D1, D2, D3, and D4 are input to the output value calculation unit 223 of pixel A. Based on these, the output value calculation unit 223 of pixel A performs the calculation of the following equation (4).
number
[0092] In equation (4), Aout is the output value of the output value calculation unit 223 for pixel A, and Ain, B1in, B2in, C1in, C2in, D1in, D2in, D3in, and D4in are output values of the corresponding pixels.
[0093] The first term of equation (4) is the count value of pixel A detected in the first subframe period. The second term of equation (4) is an interpolated value corresponding to the count value of pixel A from the second subframe period to the fourth subframe period. As shown in equation (4), output value calculation unit 223 adds the first term, which is the count value of pixel A detected in the first subframe period, to the second term, which is the interpolated value, and outputs the sum. As shown in equation (4), this interpolated value is calculated based on the output values of pixels A, B1, B2, C1, C2, D1, D2, D3, and D4.
[0094] According to the present embodiment, a photoelectric conversion device capable of improving the detection accuracy of incident light is provided, similarly to the first to third embodiments. Moreover, in the present embodiment, the period of the non-detection state can be made longer than in the first to third embodiments, so that power consumption can be further reduced.
[0095] Although equation (4) shows a simple interpolation method in which the weighting coefficients are 1 / 2 and 1 / 4, the weighting ratio of each pixel may be made different by using an interpolation parameter as in the first to third embodiments. Also, as in the second and third embodiments, a parameter calculation unit 225 may be further provided that calculates the value of an interpolation parameter according to the output value of each pixel.
[0096] [Fifth embodiment] A photoelectric conversion device 100 according to a fifth embodiment will be described with reference to Fig. 16. In the description of this embodiment, elements common to the first to fourth embodiments may be omitted. This embodiment differs from the first to fourth embodiments in that the number of pixels included in a pixel block BL varies depending on the position within the pixel region 12 or the first circuit region 22.
[0097] FIG. 16 is a diagram showing an arrangement of pixel blocks BL in the photoelectric conversion device 100 according to this embodiment. The outer frame of FIG. 16 is a schematic plan view of the range of the pixel region 12 or the first circuit region 22. In FIG. 16, the region marked "1×2" indicates a region of a pixel block BL composed of two pixels in two rows and one column, as in the first, second or third embodiment. In FIG. 16, the region marked "2×2" indicates a region of a pixel block BL composed of four pixels in two rows and two columns, as in the fourth embodiment. The pixels in the region marked "1×2" or "2×2" switch between a detection state and a non-detection state in a plurality of subframe periods within one frame period, as in the first to fourth embodiments described above. In FIG. 16, in the pixels not marked "1×2" or "2×2", no subframe period of the non-detection state is provided within one frame period, and the detection state is maintained.
[0098] Generally, in a photoelectric conversion element that counts incident light, such as an avalanche photodiode, the power consumption increases as the number of counts increases. Therefore, it is desirable to reduce power consumption by arranging a pixel block BL with a large number of pixels, such as 2 rows and 2 columns, in a portion of a strong light amount according to the amount of incident light in the pixel region and applying a driving method such as that of the fourth embodiment. It is also desirable to arrange a pixel block BL with an intermediate number of pixels, such as 2 rows and 1 column, in a portion of an intermediate light amount and apply a driving method such as that of the first to third embodiments, thereby achieving both reduction in power consumption and improvement in detection accuracy. It is also desirable to apply a driving method that does not cause a non-detection state to a portion of a weak light amount, thereby ensuring sufficient detection accuracy.
[0099] According to this embodiment, a photoelectric conversion device is provided that can obtain the same effects as any one of the first to fourth embodiments. Also, in this embodiment, by varying the number of pixels included in the pixel block BL depending on the position in the pixel region 12 or the first circuit region 22, it is possible to perform driving that takes into consideration the balance between improving the detection accuracy of incident light and reducing power consumption.
[0100] [Sixth embodiment] A light detection system according to a sixth embodiment of the present invention will be described with reference to Fig. 17. Fig. 17 is a block diagram of the light detection system according to this embodiment. The light detection system according to this embodiment is an imaging system that acquires an image based on incident light.
[0101] The photoelectric conversion device in the above-described embodiment can be applied to various imaging systems. Examples of imaging systems include digital still cameras, digital camcorders, camera heads, copiers, fax machines, mobile phones, vehicle-mounted cameras, observation satellites, and surveillance cameras. Fig. 17 shows a block diagram of a digital still camera as an example of an imaging system.
[0102] The imaging system 7 shown in Fig. 17 includes a barrier 706, a lens 702, an aperture 704, an imaging device 70, a signal processing unit 708, a timing generating unit 720, an overall control and calculation unit 718, a memory unit 710, a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. The barrier 706 protects the lens, and the lens 702 forms an optical image of a subject on the imaging device 70. The aperture 704 varies the amount of light that passes through the lens 702. The imaging device 70 is configured like the photoelectric conversion device of the above-mentioned embodiment, and converts the optical image formed by the lens 702 into image data. The signal processing unit 708 performs various types of correction, data compression, and other processes on the imaging data output from the imaging device 70.
[0103] The timing generating unit 720 outputs various timing signals to the imaging device 70 and the signal processing unit 708. The overall control / calculation unit 718 controls the entire digital still camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to the recording medium 714, and the recording medium 714 is a removable recording medium such as a semiconductor memory for recording or reading image data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the imaging system 7, and the imaging system 7 only needs to have at least the imaging device 70 and a signal processing unit 708 for processing image signals output from the imaging device 70.
[0104] In this embodiment, the imaging device 70 and the signal processing unit 708 may be disposed on the same semiconductor substrate, or the imaging device 70 and the signal processing unit 708 may be disposed on separate semiconductor substrates.
[0105] Furthermore, each pixel of the imaging device 70 may include a first photoelectric conversion unit and a second photoelectric conversion unit. The signal processing unit 708 processes a pixel signal based on the charge generated in the first photoelectric conversion unit and a pixel signal based on the charge generated in the second photoelectric conversion unit, and can acquire distance information from the imaging device 70 to the subject.
[0106] [Seventh embodiment] Fig. 18 is a block diagram of a light detection system according to this embodiment, more specifically, a block diagram of a range image sensor using the photoelectric conversion device described in the above embodiment.
[0107] 18, the range image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The range image sensor 401 receives light (modulated light, pulsed light) emitted from a light source device 411 toward a subject and reflected from the surface of the subject. The range image sensor 401 can obtain a range image according to the distance to the subject, based on the time from emission to reception of light.
[0108] The optical system 402 includes one or more lenses, guides image light (incident light) from a subject to the photoelectric conversion device 403 , and forms an image on the light receiving surface (sensor portion) of the photoelectric conversion device 403 .
[0109] The photoelectric conversion device according to each of the above-mentioned embodiments can be applied as the photoelectric conversion device 403. The photoelectric conversion device 403 supplies the image processing circuit 404 with a distance signal indicating a distance determined from a received light signal.
[0110] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by the image processing can be displayed on a monitor 405 and stored (recorded) in a memory 406.
[0111] The range image sensor 401 configured in this manner can obtain an accurate range image by applying the above-mentioned photoelectric conversion device.
[0112] [Eighth embodiment] The technology according to the present disclosure may be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system, which is an example of an optical detection system.
[0113] Fig. 19 is a schematic diagram of the endoscopic surgery system in this embodiment. Fig. 19 shows a state in which an operator (doctor) 1131 performs surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 includes an endoscope 1100, a surgical tool 1110, an arm 1121, and a cart 1134 on which various devices for endoscopic surgery are mounted.
[0114] The endoscope 1100 includes a lens barrel 1101, the region of which having a predetermined length from the tip is inserted into a body cavity of a patient 1132, and a camera head 1102 connected to the base end of the lens barrel 1101. Although Fig. 19 shows the endoscope 1100 configured as a so-called rigid scope having a rigid lens barrel 1101, the endoscope 1100 may also be configured as a so-called flexible scope having a flexible lens barrel.
[0115] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100. Light generated by the light source device 1203 is guided to the tip of the lens barrel 1101 by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
[0116] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is focused on the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. As the photoelectric conversion device, the photoelectric conversion device described in each of the above-mentioned embodiments may be used. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.
[0117] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.
[0118] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .
[0119] The light source device 1203 includes a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when imaging an operation site or the like.
[0120] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.
[0121] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.
[0122] The light source device 1203 can supply irradiation light to the endoscope 1100 when imaging the surgical site, and can be, for example, a white light source made of an LED, a laser light source, or a combination of these. When a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. Therefore, the light source device 1203 can adjust the white balance of the captured image. In this case, the laser light from each of the RGB laser light sources can be irradiated to the observation target in a time-division manner, and the drive of the image sensor of the camera head 1102 can be controlled in synchronization with the irradiation timing. This makes it possible to capture images corresponding to each of the RGB colors in a time-division manner. According to this method, a color image can be obtained without providing a color filter to the image sensor.
[0123] Furthermore, the driving of the light source device 1203 may be controlled so that the intensity of the light output from the light source device 1203 is changed at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.
[0124] Furthermore, the light source device 1203 may be configured to be capable of supplying light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue can be utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface can be photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) during normal observation. Alternatively, in the special light observation, a fluorescent observation may be performed in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to be capable of supplying narrow band light and / or excitation light corresponding to such special light observation.
[0125] [Ninth embodiment] The light detection system and the moving body of this embodiment will be described with reference to Figures 20, 21(a), 21(b), 21(c), and 22. In this embodiment, an example of an in-vehicle camera will be shown as the light detection system.
[0126] FIG. 20 is a schematic diagram of a light detection system according to this embodiment, showing an example of a vehicle system and a light detection system mounted on the vehicle system. The light detection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any of the photoelectric conversion devices according to the above-mentioned embodiments. The image preprocessing unit 1315 performs predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated in the photoelectric conversion device 1302. The light detection system 1301 is provided with at least two sets of an optical system 1314 , a photoelectric conversion device 1302 , and an image pre-processing unit 1315 , and the output from each set of the image pre-processing unit 1315 is input to the integrated circuit 1303 .
[0127] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a storage medium 1305, an optical distance measuring unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 1315. The storage medium 1305 performs primary storage of the captured image and stores the defective positions of the captured pixels. The optical distance measuring unit 1306 performs focusing or distance measurement of the subject. The parallax calculation unit 1307 calculates distance measurement information from multiple image data acquired by multiple photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it notifies the main control unit 1313 of the abnormality.
[0128] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.
[0129] The main control unit 1313 supervises and controls the operations of the light detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. The main control unit 1313 may not be provided, and the light detection system 1301, the vehicle sensor 1310, and the control unit 1320 may each have a communication interface and transmit and receive control signals via a communication network according to, for example, the CAN standard.
[0130] The integrated circuit 1303 has a function of receiving a control signal from a main control unit 1313 or transmitting a control signal or a set value to the photoelectric conversion device 1302 by its own control unit.
[0131] The optical detection system 1301 is connected to a vehicle sensor 1310, and can detect the vehicle's driving state such as vehicle speed, yaw rate, steering angle, etc., the environment outside the vehicle, and the state of other vehicles and obstacles. The vehicle sensor 1310 is also a distance information acquisition unit that acquires distance information to an object. The optical detection system 1301 is also connected to a driving assistance control unit 1311 that performs various driving assistance functions such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 1301 and the vehicle sensor 1310. This allows for avoidance control when a collision is estimated, and activation of a safety device when a collision occurs.
[0132] The light detection system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control such as applying the brakes, releasing the accelerator, and suppressing engine output to avoid a collision or reduce damage. The alarm device 1312 issues an alarm to the user using means such as issuing an alarm such as a sound, displaying alarm information on the display screen of a car navigation system or a meter panel, or applying vibration to a seat belt or steering wheel.
[0133] The light detection system 1301 in this embodiment can capture images of the surroundings of the vehicle, for example, the front or rear. Figures 21(a), 21(b), and 21(c) are schematic diagrams of a moving body in this embodiment, showing a configuration in which the light detection system 1301 captures an image of the area in front of the vehicle.
[0134] The two photoelectric conversion devices 1302 are disposed in front of the vehicle 1300. Specifically, it is preferable to regard the center line of the vehicle 1300 relative to the forward / backward direction or the outer shape (e.g., vehicle width) as an axis of symmetry, and to dispose the two photoelectric conversion devices 1302 in line symmetry with respect to the axis of symmetry. This makes it possible to effectively obtain distance information between the vehicle 1300 and an object to be photographed, and to determine the possibility of a collision. It is also preferable that the photoelectric conversion device 1302 is disposed in a position that does not obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. It is preferable that the alarm device 1312 is disposed in a position that is easily within the driver's field of vision.
[0135] Next, a fault detection operation of the photoelectric conversion device 1302 in the photodetection system 1301 will be described with reference to Fig. 22. Fig. 22 is a flowchart showing the operation of the photodetection system in this embodiment. The fault detection operation of the photoelectric conversion device 1302 can be performed in accordance with steps S1410 to S1480 shown in Fig. 22.
[0136] In step S1410, startup settings are made for the photoelectric conversion device 1302. That is, setting information for the operation of the photoelectric conversion device 1302 is transmitted from outside the photodetection system 1301 (e.g., the main control unit 1313) or from inside the photodetection system 1301, and the photoelectric conversion device 1302 starts imaging operation and fault detection operation.
[0137] Next, in step S1420, the photoelectric conversion device 1302 acquires pixel signals from the valid pixels. In addition, in step S1430, the photoelectric conversion device 1302 acquires output values from failure detection pixels provided for failure detection. The failure detection pixels include photoelectric conversion elements, like the valid pixels. A predetermined voltage is written to the photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to the photoelectric conversion elements. Note that steps S1420 and S1430 may be executed in the reverse order.
[0138] Next, in step S1440, the photodetection system 1301 performs a correspondence determination between the output expected value of the fault detection pixel and the actual output value from the fault detection pixel. If the correspondence determination result in step S1440 indicates that the output expected value and the actual output value match, the photodetection system 1301 proceeds to processing in step S1450, determines that the imaging operation is performed normally, and proceeds to processing in step S1460. In step S1460, the photodetection system 1301 transmits the pixel signal of the scanning row to the storage medium 1305 and temporarily stores it. Thereafter, the photodetection system 1301 returns to the processing in step S1420 and continues the fault detection operation. On the other hand, if the correspondence determination result in step S1440 indicates that the output expected value and the actual output value do not match, the photodetection system 1301 proceeds to processing in step S1470. In step S1470, the light detection system 1301 determines that there is an abnormality in the imaging operation, and issues an alarm to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays on the display unit that an abnormality has been detected. After that, in step S1480, the light detection system 1301 stops the photoelectric conversion device 1302, and ends the operation of the light detection system 1301.
[0139] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.
[0140] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles, automatic driving control not to go out of the lane, etc. Furthermore, the light detection system 1301 is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the light detection system 1301 is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).
[0141] The photoelectric conversion device of the present invention may further be configured to be capable of acquiring various types of information such as distance information.
[0142] [Tenth embodiment] FIG. 23(a) is a diagram showing a specific example of an electronic device in this embodiment, showing glasses 1600 (smart glasses). The glasses 1600 are provided with a photoelectric conversion device 1602 described in each of the above-mentioned embodiments. That is, the glasses 1600 are an example of a light detection system to which the photoelectric conversion device 1602 described in each of the above-mentioned embodiments can be applied. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. In addition, multiple types of photoelectric conversion devices may be combined. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 23(a).
[0143] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the above-mentioned display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is provided with an optical system for collecting light on the photoelectric conversion device 1602.
[0144] FIG. 23(b) shows glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The photoelectric conversion device in the control device 1612 and an optical system for projecting light emitted from the display device are arranged in the lens 1611, and an image is projected onto the lens 1611. The control device 1612 functions as a power source that supplies power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device 1612 may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used to detect the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An image of the eyeball is obtained by detecting the reflected light of the emitted infrared light from the eyeball with an imaging unit having a light receiving element. By providing a reduction unit that reduces the amount of light from the infrared light emitting unit to the display unit in a plan view, degradation of image quality is reduced.
[0145] The control device 1612 detects the user's line of sight with respect to the displayed image from the captured image of the eyeball obtained by capturing infrared light. Any known method can be applied to the line of sight detection using the captured image of the eyeball. As an example, a line of sight detection method based on the Purkinje image formed by the reflection of irradiated light on the cornea can be used.
[0146] More specifically, a gaze detection process based on the pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.
[0147] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information on the line of sight of a user from the photoelectric conversion device.
[0148] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. In the display area of the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.
[0149] The display area may include a first display area and a second display area different from the first display area. A high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.
[0150] In addition, AI (Artificial Intelligence) may be used in determining the first field of view area and the area with high priority. The AI may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using an image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI program may be provided in either the display device or the photoelectric conversion device, or may be provided in an external device. If the external device has the AI program, it may be transmitted to the display device from a server or the like via communication.
[0151] In the case where display control is performed based on visual recognition detection, the present embodiment can be preferably applied to smart glasses further including a photoelectric conversion device that captures an image of the outside world. The smart glasses can display captured outside information in real time.
[0152] [Modified embodiment] The present invention is not limited to the above-described embodiments and can be modified in various ways. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.
[0153] The disclosure of this specification includes the complement of the concepts described in this specification. In other words, if the specification states, for example, that "A is B" (A=B), the specification is deemed to disclose or suggest that "A is not B" even if the statement that "A is not B" (A≠B) is omitted. This is because when it states that "A is B," it is assumed that the case that "A is not B" is taken into consideration.
[0154] The disclosure of this specification includes the following configurations. (Configuration 1) A plurality of pixels each including a photoelectric conversion unit and a count unit that generates a count value based on light incident on the photoelectric conversion unit; A calculation unit that performs an interpolation process on the count value; having Each of the plurality of pixels is controlled to be in either a first state in which generation of the count value based on the incident light is enabled or a second state in which generation of the count value based on the incident light is disabled; the plurality of pixels includes a first pixel and a second pixel, In a first subframe period within one frame period, the first pixel is controlled to the first state and the second pixel is controlled to the second state; In a second subframe period within the one frame period, the second pixel is controlled to the first state and the first pixel is controlled to the second state; The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a first count value generated in the second pixel in the second sub-frame period. A photoelectric conversion device comprising: (Configuration 2) the plurality of pixels includes a plurality of the second pixels, The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a plurality of first count values generated in the plurality of second pixels, respectively, in the second sub-frame period. 2. The photoelectric conversion device according to configuration 1. (Configuration 3) the plurality of pixels includes four or more of the second pixels, The calculation unit calculates second count values of the first pixels in the second subframe period by the interpolation process using four or more first count values generated in four or more of the second pixels in the second subframe period. 3. The photoelectric conversion device according to configuration 1 or 2. (Configuration 4) The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a third count value generated in the first pixel in the first sub-frame period and a first count value generated in the second pixel in the second sub-frame period. 4. The photoelectric conversion device according to any one of configurations 1 to 3. (Configuration 5) The interpolation process includes performing a weighted addition of a third count value generated in the first pixel during the first sub-frame period and a first count value generated in the second pixel during the second sub-frame period. 5. The photoelectric conversion device according to configuration 4. (Configuration 6) The calculation unit calculates an interpolation parameter to be used in the interpolation process by using a third count value generated in the first sub-frame period at the first pixel and a first count value generated in the second pixel at the second sub-frame period. 6. The photoelectric conversion device according to any one of configurations 1 to 5. (Configuration 7) The calculation units are arranged corresponding to each of the plurality of pixels. 7. The photoelectric conversion device according to any one of configurations 1 to 6. (Configuration 8) The pixels are arranged in a plurality of rows and a plurality of columns, The calculation unit is arranged so that signals from a plurality of pixels in one column are input via a common signal line. 7. The photoelectric conversion device according to any one of configurations 1 to 6. (Configuration 9) the plurality of pixels further includes a third pixel and a fourth pixel; the one frame period includes the first subframe period, the second subframe period, the third subframe period, and a fourth subframe period, In each of the first subframe period, the second subframe period, the third subframe period, and the fourth subframe period, a portion of the first pixel, the second pixel, the third pixel, and the fourth pixel are controlled to the first state, and another portion of the first pixel, the second pixel, the third pixel, and the fourth pixel are controlled to the second state. 9. The photoelectric conversion device according to any one of configurations 1 to 8. (Configuration 10) In each of the first subframe period, the second subframe period, the third subframe period, and the fourth subframe period, one of the first pixel, the second pixel, the third pixel, and the fourth pixel is controlled to the first state, and the other three of the first pixel, the second pixel, the third pixel, and the fourth pixel are controlled to the second state. 10. The photoelectric conversion device according to configuration 9. (Configuration 11) In the first subframe period, the first pixel is controlled to the first state, and the second pixel, the third pixel, and the fourth pixel are controlled to the second state; In the second subframe period, the second pixel is controlled to the first state, and the first pixel, the third pixel, and the fourth pixel are controlled to the second state; In the third subframe period, the third pixel is controlled to the first state, and the first pixel, the second pixel, and the fourth pixel are controlled to the second state; In the fourth subframe period, the fourth pixel is controlled to the first state, and the first pixel, the second pixel, and the third pixel are controlled to the second state. 11. The photoelectric conversion device according to configuration 9 or 10. (Configuration 12) The calculation unit calculates a second count value of the first pixel from the second subframe period to the fourth subframe period by the interpolation process using a first count value generated in the second pixel during the second subframe period, a fourth count value generated in the third pixel during the third subframe period, and a fifth count value generated in the fourth pixel during the fourth subframe period. 12. The photoelectric conversion device according to claim 11, (Configuration 13) the plurality of pixels includes a plurality of pixel blocks each including at least the first pixel and the second pixel, The number of pixels included in two of the plurality of pixel blocks is different from each other. 13. The photoelectric conversion device according to any one of configurations 1 to 12. (Configuration 14) The pixels are arranged in a plurality of rows and a plurality of columns, The first pixel and the second pixel are adjacent in either a row or a column direction. 14. The photoelectric conversion device according to any one of configurations 1 to 13. (Configuration 15) Each of the plurality of pixels further includes a transistor that is connected to the photoelectric conversion unit and controls the corresponding pixel to be in the first state or the second state. 15. The photoelectric conversion device according to any one of configurations 1 to 14. (Configuration 16) Each of the plurality of pixels further includes a control unit that controls the transistor by supplying a potential to a gate of the transistor. 16. The photoelectric conversion device according to configuration 15. (Configuration 17) The calculation unit adds the second count value calculated by the interpolation process and a third count value generated in the first pixel during the first sub-frame period, and outputs the sum. 17. The photoelectric conversion device according to any one of configurations 1 to 16, (Configuration 18) the photoelectric conversion unit is an avalanche photodiode, The counting unit counts photons incident on the photoelectric conversion unit to generate the count value. 18. The photoelectric conversion device according to any one of configurations 1 to 17. (Configuration 19) The photoelectric conversion device according to any one of configurations 1 to 18, a signal processing unit that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising: (Configuration 20) A mobile object, The photoelectric conversion device according to any one of configurations 1 to 18, a distance information acquisition unit that acquires distance information to an object from a signal output from the photoelectric conversion device; A control unit that controls the moving object based on the distance information; A moving object comprising:
[0155] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.
[0156] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]
[0157] 201 Avalanche photodiode 211 Counting Department 221 Counter 222 Memory 223 Output value calculation unit
Claims
1. A plurality of pixels each including a photoelectric conversion unit and a count unit that generates a count value based on light incident on the photoelectric conversion unit; A calculation unit that performs an interpolation process on the count value; having Each of the plurality of pixels is controlled to be in either a first state in which generation of the count value based on the incident light is enabled or a second state in which generation of the count value based on the incident light is disabled; the plurality of pixels includes a first pixel and a second pixel, In a first subframe period within one frame period, the first pixel is controlled to the first state and the second pixel is controlled to the second state; In a second subframe period within the one frame period, the second pixel is controlled to the first state and the first pixel is controlled to the second state; The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a first count value generated in the second pixel in the second sub-frame period. A photoelectric conversion device comprising:
2. the plurality of pixels includes a plurality of the second pixels, The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a plurality of first count values generated in the plurality of second pixels, respectively, in the second sub-frame period.
2. The photoelectric conversion device according to claim 1.
3. the plurality of pixels includes four or more of the second pixels, The calculation unit calculates second count values of the first pixels in the second subframe period by the interpolation process using four or more first count values generated in four or more of the second pixels in the second subframe period.
2. The photoelectric conversion device according to claim 1.
4. The calculation unit calculates a second count value of the first pixel in the second sub-frame period by the interpolation process using a third count value generated in the first pixel in the first sub-frame period and a first count value generated in the second pixel in the second sub-frame period.
2. The photoelectric conversion device according to claim 1.
5. The interpolation process includes performing a weighted addition of a third count value generated in the first pixel during the first sub-frame period and a first count value generated in the second pixel during the second sub-frame period.
5. The photoelectric conversion device according to claim 4.
6. The calculation unit calculates an interpolation parameter to be used in the interpolation process by using a third count value generated in the first pixel during the first sub-frame period and a first count value generated in the second pixel during the second sub-frame period.
2. The photoelectric conversion device according to claim 1.
7. The calculation units are arranged corresponding to each of the plurality of pixels.
2. The photoelectric conversion device according to claim 1.
8. The pixels are arranged in a plurality of rows and a plurality of columns, The calculation unit is arranged so that signals are input from a plurality of pixels in one column via a common signal line.
2. The photoelectric conversion device according to claim 1.
9. the plurality of pixels further includes a third pixel and a fourth pixel; the one frame period includes the first subframe period, the second subframe period, the third subframe period, and a fourth subframe period, In each of the first subframe period, the second subframe period, the third subframe period, and the fourth subframe period, a portion of the first pixel, the second pixel, the third pixel, and the fourth pixel is controlled to the first state, and another portion of the first pixel, the second pixel, the third pixel, and the fourth pixel is controlled to the second state.
2. The photoelectric conversion device according to claim 1.
10. In each of the first subframe period, the second subframe period, the third subframe period, and the fourth subframe period, one of the first pixel, the second pixel, the third pixel, and the fourth pixel is controlled to the first state, and the other three of the first pixel, the second pixel, the third pixel, and the fourth pixel are controlled to the second state.
10. The photoelectric conversion device according to claim 9.
11. In the first subframe period, the first pixel is controlled to the first state, and the second pixel, the third pixel, and the fourth pixel are controlled to the second state; In the second subframe period, the second pixel is controlled to the first state, and the first pixel, the third pixel, and the fourth pixel are controlled to the second state; In the third subframe period, the third pixel is controlled to the first state, and the first pixel, the second pixel, and the fourth pixel are controlled to the second state; In the fourth subframe period, the fourth pixel is controlled to the first state, and the first pixel, the second pixel, and the third pixel are controlled to the second state.
10. The photoelectric conversion device according to claim 9.
12. The calculation unit calculates a second count value of the first pixel from the second subframe period to the fourth subframe period by the interpolation process using a first count value generated in the second pixel during the second subframe period, a fourth count value generated in the third pixel during the third subframe period, and a fifth count value generated in the fourth pixel during the fourth subframe period. The photoelectric conversion device according to claim 11 .
13. the plurality of pixels includes a plurality of pixel blocks each including at least the first pixel and the second pixel, The number of pixels included in two of the plurality of pixel blocks is different from each other.
2. The photoelectric conversion device according to claim 1.
14. The pixels are arranged in a plurality of rows and a plurality of columns, The first pixel and the second pixel are adjacent in either a row or a column direction.
2. The photoelectric conversion device according to claim 1.
15. Each of the plurality of pixels further includes a transistor that is connected to the photoelectric conversion unit and controls the corresponding pixel to be in the first state or the second state.
2. The photoelectric conversion device according to claim 1.
16. Each of the plurality of pixels further includes a control unit that controls the transistor by supplying a potential to a gate of the transistor.
16. The photoelectric conversion device according to claim 15.
17. The calculation unit adds the second count value calculated by the interpolation process and a third count value generated in the first pixel during the first sub-frame period, and outputs the result.
2. The photoelectric conversion device according to claim 1.
18. the photoelectric conversion unit is an avalanche photodiode, The counting unit counts photons incident on the photoelectric conversion unit to generate the count value.
2. The photoelectric conversion device according to claim 1.
19. The photoelectric conversion device according to any one of claims 1 to 18, a signal processing unit that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising:
20. A mobile object, The photoelectric conversion device according to any one of claims 1 to 18, a distance information acquisition unit that acquires distance information to an object from a signal output from the photoelectric conversion device; A control unit that controls the moving object based on the distance information; A moving object comprising:
Citation Information
Patent Citations
Imaging apparatus and control method of the same, program, and storage medium
JP2020028081A
