Imaging apparatus, imaging system, and movable body
By implementing a selective signal readout mechanism in photon counting imaging devices, the device addresses high power consumption and slow readout speed issues, achieving improved efficiency and speed in signal processing.
Patent Information
- Application Number
- JP2024161922
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-12-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing photon counting type imaging devices using avalanche diodes face issues with high power consumption and slow readout speed due to continuous avalanche multiplication in photoelectric conversion elements, leading to inefficient signal reading.
The imaging device employs a selection mechanism to control the readout of signals from photoelectric conversion elements, allowing for selective activation and deactivation of signal reading based on modes, thereby reducing unnecessary power consumption and improving readout speed.
This approach enhances the imaging device's performance by reducing power consumption and accelerating signal readout times while maintaining image quality.
Smart Images

Figure 2024169589000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging device, an imaging system, and a moving object. [Background technology]
[0002] Photon-counting imaging devices using avalanche diodes are known. Patent Document 1 discloses an imaging device in which a plurality of photoelectric conversion elements including avalanche diodes are arranged in a matrix. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] US Patent Application Publication No. 2015 / 0115131 Summary of the Invention [Problem to be solved by the invention]
[0004] The imaging device described in Patent Document 1 has room for improvement in characteristics such as power consumption and readout speed. In the imaging device described in Patent Document 1, if an attempt is made to constantly cause avalanche multiplication in each photoelectric conversion element to read out signals from the photoelectric conversion elements, power consumption increases and it takes time to finish reading out signals from all the photoelectric conversion elements. [Means for solving the problem]
[0005] An imaging device according to one aspect of the present invention comprises a plurality of photoelectric conversion elements, each of which includes an avalanche diode and a counter that counts a signal based on light incident on the avalanche diode, a first selection means is provided corresponding to a first photoelectric conversion element among the plurality of photoelectric conversion elements, and a second selection means is provided corresponding to a second photoelectric conversion element among the plurality of photoelectric conversion elements, and in a first mode, the first selection means controls the first photoelectric conversion element to a state in which a signal is read out, and the second selection means controls the second photoelectric conversion element to a state in which a signal is not read out, and in a second mode, which is set for a period different from the period in which the first mode is set, the second selection means controls the second photoelectric conversion element, which was controlled to a state in which the signal is not read out, to a state in which a signal is read out. Effect of the Invention
[0006] In a photon-counting type imaging device using an avalanche diode, the characteristics of the imaging device, such as power consumption and readout speed, can be improved. [Brief description of the drawings]
[0007] [Figure 1] 1 is a schematic diagram of an example of a photon-counting imaging device using an avalanche diode. [Diagram 2] 1 is an equivalent circuit diagram of a photoelectric conversion element 11 according to the first embodiment. [Diagram 3] 5A to 5C are diagrams illustrating an example of a photon detection operation of the photoelectric conversion element 11 according to the first embodiment. [Figure 4] 2 is a conceptual diagram of a readout region of the pixel region 10 according to the first embodiment. FIG. [Diagram 5] FIG. 11 is a diagram showing an equivalent circuit diagram of a photoelectric conversion element 11 according to a second embodiment. [Figure 6] FIG. 11 is a diagram showing another example of an equivalent circuit diagram of the photoelectric conversion element 11 according to the second embodiment. [Figure 7] FIG. 11 is a diagram showing an equivalent circuit diagram of a photoelectric conversion element 11 according to a third embodiment. [Figure 8] 10 is a block diagram showing a part of a photoelectric conversion element 11 according to a fourth embodiment. FIG. [Figure 9] FIG. 13 is a schematic diagram of the structure of an imaging device according to a fifth embodiment. [Figure 10] FIG. 13 is a block diagram showing the configuration of an imaging system according to a sixth embodiment. [Figure 11] FIG. 13 is a schematic diagram showing an example of the configuration of an imaging system and a moving object according to a seventh embodiment. [Figure 12] FIG. 13 is a flowchart showing the operation of the imaging system according to the seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] (First embodiment) An imaging device according to a first embodiment will be described with reference to Figs. 1 to 4. Fig. 1 is a block diagram showing a schematic configuration of an imaging device according to this embodiment. Fig. 2 is a circuit diagram showing a configuration example of a photoelectric conversion element 11 of an imaging device according to this embodiment. In the following, a case where one photoelectric conversion element constitutes one pixel will be described, but the present invention also includes a case where a plurality of photoelectric conversion elements constitute one pixel.
[0009] 1, the imaging device includes a pixel region 10, a vertical scanning circuit 20, a horizontal scanning circuit 30, an output circuit 40, and a control circuit 50. The pixel region 10 is provided with a plurality of photoelectric conversion elements 11 arranged in a matrix across a plurality of rows and columns. It is not necessary for all components of the photoelectric conversion elements 11 to be arranged in a plurality of rows and columns. It is sufficient that at least the avalanche diodes included in the photoelectric conversion elements 11 are arranged in a plurality of rows and columns.
[0010] A control line 12 is arranged in each row of the pixel region 10, extending in a first direction (the horizontal direction in FIG. 1). The control line 12 is connected to each of the photoelectric conversion elements 11 arranged in the first direction, and serves as a signal line common to these photoelectric conversion elements 11. The first direction in which the control lines 12 extend may be referred to as the row direction or the horizontal direction. Each of the control lines 12 may include a plurality of signal lines for supplying a plurality of types of control signals to the photoelectric conversion elements 11.
[0011] Furthermore, in the pixel region 10, a control line VSEL is arranged, extending in the first direction and connected to the selection means included in the photoelectric conversion element 11. In Figures 1 and 2, each photoelectric conversion element 11 includes a selection means, and therefore a control line VSEL is arranged in each row. However, this is not limiting, and in the case where the selection means is shared by a plurality of photoelectric conversion elements, the control line VSEL may be arranged corresponding to the photoelectric conversion element 11 in which the selection means is arranged.
[0012] In each column of the pixel region 10, a data line 13 is arranged to extend in a second direction (vertical direction in FIG. 1) intersecting the first direction. The data line 13 is connected to each of the photoelectric conversion elements 11 arranged in the second direction, and serves as a signal line common to these photoelectric conversion elements 11. The second direction in which the data lines 13 extend may be referred to as a column direction or a vertical direction. Each of the data lines 13 may include a plurality of signal lines for transferring digital signals output from the photoelectric conversion elements 11 on a bit-by-bit basis.
[0013] Furthermore, in the pixel region 10, an HSEL is arranged extending in the second direction and connected to a selection means included in the photoelectric conversion element 11. In Figures 1 and 2, each photoelectric conversion element 11 includes a selection means, and therefore a control line HSEL is arranged in each column. However, without being limited to this, when the selection means is shared by a plurality of photoelectric conversion elements, it is sufficient that the control line HSEL is arranged corresponding to the photoelectric conversion element 11 in which the selection means is arranged.
[0014] The control lines 12 and the control lines VSEL of each row are connected to a vertical scanning circuit 20. The vertical scanning circuit 20 is a circuit section that supplies control signals for reading out signals from the photoelectric conversion elements 11 to the photoelectric conversion elements 11 via the control lines 12.
[0015] The control line 12 is, for example, a control line that controls a signal processing circuit SP included in the photoelectric conversion element 11. The vertical scanning circuit 20 sequentially scans the photoelectric conversion elements 11 in the pixel region 10 on a row-by-row basis, and outputs a signal of each photoelectric conversion element 11 to the output circuit 40 via the data line 13. In addition, the control line VSEL is a line that controls a selection means included in the photoelectric conversion element 11.
[0016] The data line 13 of each column is connected to a horizontal scanning circuit 30. The control line HSEL of each column is connected to the horizontal scanning circuit 30. The horizontal scanning circuit 30 is a circuit section that selects the signals of the photoelectric conversion elements 11 of each column outputted row by row from the pixel region 10 and sequentially outputs the signals to the output circuit 40. The horizontal scanning circuit 30 includes a plurality of holding sections corresponding to a plurality of columns of the pixel region 10, and holds the signals of the photoelectric conversion elements 11 of each column outputted row by row from the pixel region 10 in the holding section of the corresponding column. The horizontal scanning circuit 30 sequentially scans the holding sections of each column and sequentially outputs the signals from the photoelectric conversion elements held in the holding sections of each column to the output circuit 40.
[0017] The output circuit 40 has a transmitter circuit 41, and is a circuit section that outputs a signal from the photoelectric conversion element 11, which is output from the horizontal scanning circuit 30, to the outside of the imaging device. The transmitter circuit 41 can be configured, for example, by a SerDes (SERializer / DESerializer) transmission circuit. The SerDes transmission circuit is, for example, an LVDS (Low Voltage Differential Signaling) circuit or an SLVS (Scalable Low Voltage Signaling) circuit. Note that the external interface circuit that configures the output circuit 40 is not particularly limited.
[0018] The control circuit 50 is a circuit section for supplying control signals that control the operations and timings of the vertical scanning circuit 20 and the horizontal scanning circuit 30. At least a part of the control signals that control the operations and timings of the vertical scanning circuit 20 and the horizontal scanning circuit 30 may be supplied from outside the imaging device.
[0019] As shown in FIG. 2, each photoelectric conversion element 11 has an avalanche diode D, a quench element Rq, a waveform shaping unit INV, and a signal processing circuit SP including a counter. The avalanche diode D may be configured of a photodiode that multiplies the charge generated by the incidence of photons by avalanche breakdown. Specifically, such a photodiode is an APD (Avalanche Photo Diode) or a SPAD (Single Photon Avalanche Diode). The SPAD generates avalanche multiplication in a Geiger mode in which it is operated at a bias voltage higher than the breakdown voltage. The APD generates avalanche multiplication in a linear mode in which it is operated at a bias voltage slightly higher than the breakdown voltage. The following will be described using the SPAD as an example. The avalanche diode D is connected to a counter included in the signal processing circuit SP, and the counter counts a signal based on the photons incident on the avalanche diode D. The quench element Rq may be configured of, for example, a P-type MOS transistor. The waveform shaping unit INV can be configured with, for example, an inverter circuit. The waveform shaping unit INV is connected to the counter.
[0020] The anode of the photodiode constituting the avalanche diode D is supplied with a voltage Vss, and the cathode is connected to one end of the quench element Rq and an input terminal of the waveform shaping unit INV. The other end of the quench element Rq is supplied with a voltage Vdd. The output terminal of the waveform shaping unit INV is connected to the input terminal of the counter of the signal processing circuit SP. The output terminal of the signal processing circuit SP is connected to the data line 13.
[0021] The voltages Vss and Vdd are set so that a reverse bias voltage sufficient for operating in the Geiger mode can be applied to the avalanche diode D. In one example, a negative high voltage is applied as the voltage Vss, and a positive voltage approximately equal to the power supply voltage is applied as the voltage Vdd. In this embodiment, the photodiode constituting the avalanche diode D is used as a single photon avalanche diode (SPAD) capable of operating in the Geiger mode. In this specification, a state in which a reverse bias voltage capable of avalanche multiplication is applied is also referred to as an active state, and a state in which a reverse bias voltage capable of avalanche multiplication is not applied is also referred to as an inactive state.
[0022] A reverse bias voltage equivalent to the potential difference between voltages Vdd and Vss is applied to the avalanche diode D. This reverse bias voltage is higher than the breakdown voltage of the avalanche diode D, and is sufficient to cause avalanche multiplication (Geiger mode). However, when no photons are incident on the avalanche diode D, there are no seed carriers, so avalanche multiplication does not occur and no current flows through the avalanche diode D (standby state).
[0023] When a photon is incident on the avalanche diode D in the standby state, the incident photon excites the avalanche diode D and generates carriers in the avalanche diode D. The carriers generated in the avalanche diode D are accelerated by the high electric field in the avalanche diode D, causing avalanche multiplication, and a large avalanche current is generated (Geiger mode operation). This avalanche current flows through the quench element Rq. Due to the voltage drop caused by this current, the potential of the cathode of the avalanche diode D drops, and the potential difference between the terminals of the avalanche diode D becomes smaller. As a result, the avalanche diode D goes out of Geiger mode (non-Geiger mode), and avalanche multiplication stops. The carriers on the cathode side of the avalanche diode D are gradually discharged through the quench element Rq connected as a load. In other words, the potential Vdd is supplied to the cathode of the avalanche diode D through the quench element Rq, so the potential supplied to the cathode of the avalanche diode D returns to the potential Vdd. As a result, the potential difference between the terminals of the avalanche diode D returns to the initial voltage. In other words, the avalanche diode D returns to the Geiger mode or standby state.
[0024] This series of operations will be described with reference to Fig. 3. Fig. 3 is a timing chart showing a method for driving an imaging device according to this embodiment. The timing chart in Fig. 3 shows the timing at which photons are incident on the avalanche diode D, and the signal Sig_A input to the waveform shaping unit INV. The timing chart in Fig. 3 also shows the signal Sig_D, which is a pulse-shaped signal obtained by shaping the waveform of the signal Sig_A and is output to the signal processing circuit SP, and the count value (DOUT) of the signal Sig_D by the signal processing circuit SP.
[0025] In the initial state, no photons are incident on the avalanche diode D, the signal Sig_A is at a predetermined potential indicating a standby state, the signal Sig_D is at a low level, and the count value of the signal processing circuit SP is 0.
[0026] When a photon is incident on the avalanche diode D, avalanche multiplication occurs using carriers excited by the incident photon as seeds, and an avalanche current flows, causing the potential of the signal Sig_A to decrease. When the signal Sig_A falls below a certain threshold, the potential of the signal Sig_D transitions from low to high.
[0027] Thereafter, the potential of the signal Sig_A gradually increases via the quench element Rq. When the signal Sig_A exceeds a predetermined threshold, the potential of the signal Sig_D transitions from a high level to a low level.
[0028] A counter included in the signal processing circuit SP increments a count value by 1 in accordance with the number of pulses of the signal Sig_D.
[0029] As described above, the waveform shaping unit INV generates a pulse based on the presence or absence of an avalanche current caused by avalanche multiplication. The counter included in the signal processing circuit SP counts the number of pulses based on the presence or absence of an avalanche current. The signal processing circuit SP may perform predetermined signal processing on the signal Sig_D in addition to the counting operation.
[0030] The signal processing circuit SP outputs an output signal DOUT, which is a signal after signal processing, to a data line 13 in response to a control signal 12 output from the vertical scanning circuit 20 and a control signal from a selection means 14, which will be described later.
[0031] In this embodiment, the selection means 14 switches whether or not to output a signal from the signal processing circuit SP. The control lines 12, HSEL, and VSEL are input to the signal processing circuit SP via the selection means 14. When the control lines 12, HSEL, and VSEL are all active, a signal is read from the signal processing circuit SP, and when at least one of them is inactive, a signal is not read from the signal processing circuit SP. The selection means 14 can be realized by a combinational circuit such as an AND circuit as shown in FIG. 2, but the method of realization is not limited to this.
[0032] In this embodiment as well, it is possible to read out signals only from a desired area.
[0033] In Fig. 2, the signal processing circuit SP is switched between active and inactive. Specifically, a switch is placed at a node between the signal processing circuit SP and the data line 13 to control the reading of a signal from the signal processing circuit SP to the data line. In this specification, the active state of the signal processing circuit SP refers to a state in which a signal is read out from the signal processing circuit SP. Also, the inactive state of the signal processing circuit SP refers to a state in which a signal is not read out from the signal processing circuit SP.
[0034] Fig. 4 is a diagram showing an image of a readout region 10a and a non-readout region 10b of the pixel region 10. Fig. 4 shows the region 10a that reads out signals from photoelectric conversion elements (first photoelectric conversion elements) and the region 10b that does not read out signals from photoelectric conversion elements (second photoelectric conversion elements). The region 10b shown in Fig. 4 may not always output signals, or may not output signals in the first mode and output signals in the second mode that is set in a period different from the period in which the first mode is set.
[0035] In this embodiment, photoelectric conversion is performed in the regions 10a and 10b. That is, light is also irradiated to the region 10b. In this embodiment, photoelectric conversion is also performed in the region 10b, but the region 10b is not read out from the viewpoint of improving the signal readout speed, etc. In other words, although it is not a defective pixel, whether or not to read out a signal is controlled according to the mode. Here, the mode refers to, for example, a period during which the vertical scanning circuit 20 vertically scans a plurality of photoelectric conversion elements arranged in the pixel region 10a. In FIG. 2, the period from top to bottom scanning in the first vertical scan to the end is the first vertical scan period. After the end of scanning from top to bottom and the return from bottom to top, the next period of scanning from top to bottom is the second vertical scan period. The description will be made with reference to FIG. 4(a). In this description, it is assumed that each of the first mode and the second mode reads out a signal from the region 10a. The first vertical scan period refers to a period from scanning the photoelectric conversion elements located at the upper end of the region 10a to scanning the photoelectric conversion elements located at the lower end of the region 10a. The second vertical scan refers to the period from scanning the photoelectric conversion elements located at the top end of the region 10a to finishing scanning the photoelectric conversion elements located at the bottom end of the region 10a. In this embodiment, if reading is not performed for all the photoelectric conversion elements arranged in the row or column direction, vertical or horizontal scanning is not performed either. This makes it possible to shorten the signal readout time compared to the case where signals are read out from all the photoelectric conversion elements.
[0036] For example, in FIG. 4(a), a thinning-out readout is performed in which only signals from an area of m×n pixels (m and n are natural numbers) in the pixel area 10 are read out. The area 10a and the area 10b can be controlled as shown in FIG. 4(b) to FIG. 4(d). FIG. 4(b) is an example of row thinning-out readout. Specifically, signals from photoelectric conversion elements arranged in a predetermined number of rows are read out, and signals from photoelectric conversion elements arranged in other rows are not read out. FIG. 4(c) is an example of column thinning-out readout. Specifically, signals from photoelectric conversion elements arranged in a predetermined number of columns are read out, and signals from photoelectric conversion elements arranged in other columns are not read out. FIG. 4(d) is an example of row-column thinning-out readout. Specifically, signals from photoelectric conversion elements arranged in a predetermined position are read out, and signals are not read out in rows and columns. Thinning-out may be performed in pixel units, not in matrix units as shown in FIG. 4(a) to FIG. 4(d). For example, as shown in Fig. 6, the selection means 14 includes a memory MEM, and by using the memory MEM, it is possible to perform ROI (Region Of Interest) readout as shown in Fig. 4(e). Specifically, a signal indicating whether or not to perform readout is written to the memory MEM included in each photoelectric conversion element via a signal line WRITE. Based on this signal, it may be possible to control whether to supply a potential to turn the switch on or off. In this way, the designer may realize the selection means 14 in various configurations in order to obtain the effects of the present invention.
[0037] The region 10a from which signals are read out may be different between the first mode and the second mode. For example, first, in the first mode, signals are read out from the photoelectric conversion elements arranged in the region 10a shown in FIG. 4(a) to FIG. 4(d). Then, in the second mode, signals may be read out from the photoelectric conversion elements arranged in the region 10b shown in FIG. 4(a) to FIG. 4(d), and signals may also be read out from the photoelectric conversion elements arranged in the region 10a. Also, in the first mode, signals may be read out from the region 10a, and signals may not be read out from the region 10b, and in the second mode, signals may be read out from both the region 10a and the region 10b. That is, in the second mode, the photoelectric conversion elements that were controlled to a state in which signals are not read out are controlled by the selection means (second selection means) to a state in which signals are read out. Then, the photoelectric conversion elements that were controlled to a state in which signals are read out are maintained so that they are in a state in which signals are read out. Such a readout method is effective, for example, when the imaging device captures images in a first mode until an object such as a suspicious person appears in the imaging area, and then switches from the first mode to the second mode when the object appears in the imaging area. Priority is given to the signal readout speed until the object appears, and priority is given to image quality after the object appears. As described above, it is possible to switch modes depending on the purpose of imaging.
[0038] In addition, in the ROI shown in Fig. 4(e), when capturing an image of a moving object, it is possible to switch whether or not to read out a signal depending on the position of the moving object. Specifically, in a first mode in which the moving object is located in a first portion of the region to be captured, only the first portion is read out, and in a second mode in which the moving object is located in a second portion different from the first portion, only the second portion is read out.
[0039] 2, the selection means 14 is connected to the signal processing circuit SP, but it may be connected to the input part of the waveform shaping unit INV or to a node between the waveform shaping unit INV and the signal processing circuit SP. Even in this case, the counter included in the signal processing circuit SP stops counting, and it is possible to prevent the signal from being output from the photoelectric conversion element.
[0040] In this embodiment, the photoelectric conversion elements included in the region 10a are controlled so that the signal processing circuit SP is active, and therefore a signal is output from the signal processing circuit SP. The photoelectric conversion elements included in the region 10b are controlled so that the signal processing circuit SP is inactive, and therefore no signal is output from the signal processing circuit SP. In this way, according to this embodiment, it is possible to select the photoelectric conversion elements 11 belonging to the region 10b that do not require reading. This allows the signal reading speed of the imaging device to be shorter than when all the photoelectric conversion elements are read.
[0041] Second embodiment A photoelectric conversion element 11 according to the second embodiment will be described with reference to Fig. 5. The photoelectric conversion element 11 according to the second embodiment differs from the first embodiment in that the selection means 14 is connected to the quench element Rq. Other than the matters described below, the photoelectric conversion element 11 according to the second embodiment is the same as the first embodiment.
[0042] In this embodiment, the quench element Rq is composed of a MOS transistor, and the gate terminal of the quench element Rq is connected to an output terminal of a selection means 14 that controls the on / off of the quench element Rq. The selection means 14 receives a control line VSEL arranged for each row and a control line HSEL arranged for each column. In this embodiment, the avalanche diode D is switched between active and inactive by controlling the on / off of the quench element Rq with the signal VSEL and the signal HSEL. When both the signal VSEL and the signal HSEL are active, the selection means 14 controls the quench element Rq so that the avalanche diode D becomes active. In other words, the selection means 14 controls the quench element Rq so that a reverse bias voltage that can cause avalanche multiplication is applied to the avalanche diode D. In FIG. 5, since the quench element Rq is composed of a PMOS transistor, for example, a ground voltage is applied to the gate terminal of the PMOS transistor. Furthermore, when at least one of the signals VSEL and HSEL is inactive, the selection means 14 controls the quench element Rq so that the avalanche diode D is inactive. In other words, the selection means 14 controls the avalanche diode D to have a potential at which avalanche multiplication does not occur. In FIG. 4, for example, a potential higher than Vdd is applied to the gate terminal to turn off the PMOS transistor. As a result, no potential is supplied to the cathode Vdd, and avalanche multiplication does not occur in the avalanche diode D.
[0043] When the avalanche diode D is active, an avalanche current flows every time a photon is incident, and when the avalanche diode D is inactive, no avalanche current is generated even if a photon is incident. Therefore, when the avalanche diode D is inactive, the counter of the signal processing circuit SP cannot count the signal, and no signal is read out from the photoelectric conversion element 11.
[0044] A selection circuit 14 shown in Fig. 6 may be used instead of the selection circuit 14 shown in Fig. 5. In this case, a signal line WRITE is required, so the number of signal lines increases compared to the first embodiment. In this way, the selection means 14 may be composed of a combinational circuit and a memory MEM, and the activation / inactivation of the avalanche diode D may be controlled based on a signal input to the memory MEM.
[0045] In this embodiment, the photoelectric conversion elements located in the region 10b may be scanned. Even if scanning is performed, the counter value will be 0 because no avalanche current is generated. Note that, if all the photoelectric conversion elements located in the row or column direction among the photoelectric conversion elements located in the region 10b are photoelectric conversion elements that do not read out signals, it is preferable not to perform vertical or horizontal scanning. In other words, it is preferable to perform scanning by thinning out the photoelectric conversion elements. This can improve the signal readout speed.
[0046] When the avalanche diodes D of all the photoelectric conversion elements in the pixel region 10 are active, an avalanche current flows and counting is performed every time a photon is incident on all the photoelectric conversion elements. That is, the photoelectric conversion elements 11 belonging to the region 10b that is not to be read out generate unnecessary power consumption. On the other hand, according to this embodiment, since the avalanche diodes D of the photoelectric conversion elements 11 belonging to the region 10b are inactive, even if a photon is incident, no avalanche current is generated and no counting is performed. No unnecessary power consumption is generated in the photoelectric conversion elements 11 belonging to the region 10b, and power consumption can be reduced compared to the case where signals are read out from all the photoelectric conversion elements. That is, according to this embodiment, in addition to the effects described in the first embodiment, the image pickup device can be made to consume less power. Furthermore, when vertical scanning is not performed, the signal readout speed can also be improved.
[0047] Third embodiment A photoelectric conversion element 11 according to the third embodiment will be described with reference to FIG.
[0048] In this embodiment, a switching means 15 is connected to a node different from the anode of the avalanche diode D or the node to which the avalanche diode D of the quench element Rq is connected. Also, the selection means 14 differs from the second embodiment in that it controls the switching means 15. The present embodiment is similar to the second embodiment except for the matters described below.
[0049] 7, when both the signals VSEL and HSEL are active, the selection means 14 controls the switching means 15 so that the quench element Rq is connected to the voltage Vdd, and at this time the avalanche diode D becomes active. When at least one of the signals VSEL and HSEL is inactive, the selection means 14 controls the switching means 15 so that the quench element Rq is connected to the voltage VL, and at this time the avalanche diode D becomes inactive. At this time, the voltage VL is set to an arbitrary voltage that does not cause avalanche multiplication in the avalanche diode D.
[0050] 8, a switching means 15 is provided at the anode of the avalanche diode D. When the avalanche diode D is controlled to be active, the anode is connected to the voltage Vss. When the avalanche diode D is controlled to be inactive, the anode is connected to the voltage VH. At this time, the voltage VH is set to an arbitrary voltage that does not cause avalanche amplification in the avalanche diode D.
[0051] In this manner, in this embodiment, the switching means 15 selects, via the selection means 14, the voltage line connected to the avalanche diode D between the voltage line supplying the potential that activates the avalanche diode D and the voltage line VL supplying the potential that deactivates the avalanche diode D. In this way, the activation / inactivation of the avalanche diode D is controlled.
[0052] According to this embodiment, although the number of voltage lines VL or VH is increased compared to the first embodiment, it is possible to read out signals only from a desired region as in the first embodiment. Also, as in the first embodiment, the activation / inactivation of the avalanche diode D is controlled, so that the power consumption of the imaging device can be reduced.
[0053] (Fourth embodiment) An imaging device according to the fourth embodiment will be described with reference to FIG. 9. FIG. 9 is a schematic diagram showing the structure of the imaging device according to the fourth embodiment. The imaging device in this embodiment is formed by stacking a first substrate 60b and a second substrate 60a. The first substrate 60b has a plurality of photoelectric conversion element cells (hereinafter referred to as "cells") 11d each having a signal processing circuit SP. The second substrate 60a has a plurality of cells 11c each having an avalanche diode D. The cells 11c and 11d are connected to each other. That is, one cell 11c and one cell 11d are connected to each other to form one photoelectric conversion element 11.
[0054] Here, the quench element Rq, the selection means 14, and the waveform shaping unit INV may be placed in either cell 11c or cell 11d. For example, the following arrangements are possible, and each is expected to have its own unique effect, so the designer can adopt various arrangement positions according to the purpose.
[0055] (Layout example 1) Only the avalanche diode D can be arranged in the cell 11c, and all other components can be arranged in the cell 11d. In this case, the area of the avalanche diode D is maximized, so that the aperture ratio of the photoelectric conversion element 11 can be increased. Furthermore, the second substrate 60b can be manufactured by a process specialized for the avalanche diode D, so that the pixel characteristics can be improved.
[0056] (Layout example 2) When the components of a photoelectric conversion element include transistors with different withstand voltages, only the components with low withstand voltages can be arranged in cell 11d.
[0057] For example, when the quench element Rq is composed of high-voltage transistors and the other transistors are composed of low-voltage transistors, the avalanche diode D and the quench element Rq are arranged in the cell 11c, and the other transistors are arranged in the cell 11d. Alternatively, when the output stage of the selection means 14 is also composed of high-voltage transistors for the quench element Rq composed of high-voltage transistors, the avalanche diode D, the quench element Rq, and the output stage of the selection means 14 are arranged in the cell 11c. In this way, the cell 11d can be composed of only low-voltage transistors. Therefore, the integration density of the first substrate 60b can be increased, and the first substrate 60a can be made by a process specialized for low-voltage transistors, so that good circuit characteristics can be obtained.
[0058] The arrangement of the components of the photoelectric conversion element 11 in the cells 11c and 11d in this embodiment is not limited to the above example, and any arrangement may be adopted.
[0059] According to this embodiment, in addition to the effects described in the first to third embodiments, it is possible to reduce the chip size of the imaging device while suppressing degradation of pixel characteristics.
[0060] Fifth embodiment Fig. 10 is a block diagram showing a configuration of an imaging system 200 according to this embodiment. The imaging system 200 of this embodiment includes an imaging device 204. Here, any of the imaging devices described in the above embodiments can be applied to the imaging device 204. Specific examples of the imaging system include a digital still camera, a digital camcorder, and a surveillance camera. Fig. 10 shows an example of a digital still camera as the imaging system 200.
[0061] 10 includes an imaging device 204, a lens 202 that forms an optical image of a subject on the imaging device 204, an aperture 203 that varies the amount of light passing through the lens 202, and a barrier 201 that protects the lens 202. The lens 202 and the aperture 203 form an optical system that focuses light on the imaging device 204.
[0062] The imaging system 200 has a signal processing unit 205 that processes an output signal output from an imaging device 204. The signal processing unit 205 performs signal processing operations such as performing various corrections and compression on an input signal as necessary and outputting the signal. The imaging system 200 further has a buffer memory unit 206 for temporarily storing image data, and an external interface unit (external I / F unit) 209 for communicating with an external computer or the like. The imaging system 200 further has a recording medium 211 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 210 for recording or reading out the recording medium 211. The recording medium 211 may be built in the imaging system 200 or may be removable. In addition, communication between the recording medium control I / F unit 210 and the recording medium 211 and communication from the external I / F unit 209 may be performed wirelessly.
[0063] Furthermore, the imaging system 200 has an overall control / calculation unit 208 that performs various calculations and controls the entire digital still camera, and a timing generation unit 207 that outputs various timing signals to the imaging device 204 and the signal processing unit 205. Here, the timing signals and the like may be input from outside, and the imaging system 200 only needs to have at least the imaging device 204 and the signal processing unit 205 that processes the output signal output from the imaging device 204. As described above, the control circuit 50 may control all the operations and timings of the vertical scanning circuit and the horizontal scanning circuit, or at least a part of them may be supplied from the timing generation unit 207. The overall control / calculation unit 208 and the timing generation unit 207 may be configured to implement a part or all of the control functions of the imaging device 204.
[0064] The imaging device 204 outputs an image signal to the signal processing unit 205. The signal processing unit 205 performs a predetermined signal processing on the image signal output from the imaging device 204 and outputs image data. The signal processing unit 205 generates an image using the image signal. The signal processing unit 205 may perform distance measurement calculation on the signal output from the imaging device 204. The signal processing unit 205 and the timing generating unit 207 may be mounted on the imaging device. The signal processing unit 205 and the timing generating unit 207 may be provided on a board on which a photoelectric conversion element is arranged, or may be provided on a separate board as described in the fourth embodiment. By configuring an imaging system using the imaging device of each of the above-mentioned embodiments, an imaging system capable of acquiring a better quality image can be realized.
[0065] Sixth embodiment The imaging system and the moving body of this embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a schematic diagram showing a configuration example of the imaging system and the moving body according to this embodiment. Fig. 12 is a flow diagram showing the operation of the imaging system according to this embodiment. In this embodiment, an example of an in-vehicle camera is shown as the imaging system.
[0066] FIG. 11 shows an example of a vehicle system and an imaging system mounted thereon. The imaging system 1301 includes an imaging device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the imaging device 1302. The imaging device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The imaging device 1302 is an imaging device according to any of the above-mentioned embodiments. The image preprocessing unit 1315 performs predetermined signal processing on a signal output from the imaging device 1302. The function of the image preprocessing unit 1315 may be incorporated in the imaging device 1302. The imaging system 1301 includes at least two sets of the optical system 1314, the imaging device 1302, and the image preprocessing unit 1315, and the output from each set of the image preprocessing unit 1315 is input to the integrated circuit 1303.
[0067] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a memory 1305, an optical distance measuring unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on an output signal from an image pre-processing unit 1315. The memory 1305 temporarily stores the captured image and stores the defective positions of the captured pixels. The optical distance measuring unit 1306 performs focusing and distance measurement of the subject. The parallax calculation unit 1307 calculates parallax information (phase difference of parallax images) from multiple image data acquired by multiple imaging devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the imaging device 1302, it issues an abnormality report to the main control unit 1313.
[0068] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.
[0069] The main control unit 1313 supervises and controls the operations of the imaging system 1301, the vehicle sensor 1310, the control unit 1320, etc. It is also possible to adopt a method in which the main control unit 1313 is not provided, and the imaging system 1301, the vehicle sensor 1310, and the control unit 1320 each have their own communication interface and transmit and receive control signals via a communication network (for example, CAN standard).
[0070] The integrated circuit 1303 has a function of receiving a control signal from the main control unit 1313 or transmitting a control signal or a setting value to the image pickup device 1302 by its own control unit.
[0071] The imaging system 1301 is connected to a vehicle sensor 1310, and can detect the vehicle's driving state, such as vehicle speed, yaw rate, and steering angle, as well as the state of the environment outside the vehicle and other vehicles and obstacles. The vehicle sensor 1310 is also a distance information acquisition means for acquiring distance measurement information to an object from a parallax image. The imaging system 1301 is also connected to a driving assistance control unit 1311 that performs various driving assistance functions, such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the imaging system 1301 and the vehicle sensor 1310. This allows for avoidance control when a collision is estimated, and activation of a safety device when a collision occurs.
[0072] The imaging system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 1312 warns the user by sounding an alarm, displaying alarm information on a display screen such as a car navigation system or a meter panel, vibrating a seat belt or steering wheel, etc.
[0073] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are photographed by the imaging system 1301. Fig. 12(b) shows an example of arrangement when the imaging system 1301 photographs the area in front of the vehicle.
[0074] The two imaging devices 1302 are disposed in front of the vehicle 1300. Specifically, if the center line of the vehicle 1300 with respect to its forward / backward direction or outer shape (for example, vehicle width) is regarded as an axis of symmetry, and the two imaging devices 1302 are disposed in line symmetry with respect to the axis of symmetry, this is preferable for obtaining distance information between the vehicle 1300 and an object to be photographed and for determining the possibility of a collision. In addition, the imaging devices 1302 are preferably disposed so as not to obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. The warning device 1312 is preferably disposed so as to be easily within the driver's field of vision.
[0075] Next, the operation of detecting a fault in the image capturing device 1302 in the image capturing system 1301 will be described with reference to Fig. 12. The operation of detecting a fault in the image capturing device 1302 is performed in accordance with steps S1410 to S1480 shown in Fig. 12.
[0076] Step S1410 is a step for performing startup settings of the imaging device 1302. That is, settings for the operation of the imaging device 1302 are transmitted from outside the imaging system 1301 (e.g., the main control unit 1313) or from inside the imaging system 1301, and the imaging operation and failure detection operation of the imaging device 1302 are started.
[0077] Next, in step S1420, pixel signals are obtained from the effective pixels. In addition, in step S1430, output values are obtained from failure detection pixels provided for failure detection. The failure detection pixels include avalanche diodes that function as photoelectric conversion units, just like the effective pixels. A predetermined voltage is written to the photoelectric conversion unit. The failure detection pixels output signals corresponding to the voltage written to the photoelectric conversion unit. Note that steps S1420 and S1430 may be reversed.
[0078] Next, in step S1440, a judgment is made as to whether the output expected value of the fault detection pixel corresponds to the actual output value from the fault detection pixel. If the result of the judgment in step S1440 indicates that the output expected value and the actual output value correspond, the process proceeds to step S1450, it is judged that the imaging operation is performed normally, and the process proceeds to step S1460. In step S1460, the pixel signal of the scanning row is sent to the memory 1305 and temporarily stored. Thereafter, the process returns to step S1420, and the fault detection operation is continued. On the other hand, if the result of the judgment in step S1440 indicates that the output expected value and the actual output value do not correspond, the process proceeds to step S1470. In step S1470, it is judged that there is an abnormality in the imaging operation, and an alarm is issued to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays the detection of the abnormality on the display unit. Thereafter, in step S1480, the image capturing device 1302 is stopped, and the operation of the image capturing system 1301 is terminated.
[0079] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.
[0080] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles, automatic driving control not to go out of the lane, etc. Furthermore, the imaging system 1301 is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the imaging system 1301 is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).
[0081] The imaging device of the present invention may further include a color filter and a microlens, and may be configured to be capable of acquiring various types of information such as distance information.
[0082] The present invention is not limited to the above-mentioned embodiments, and various modifications are possible. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present invention. For example, as in the first embodiment, it is possible to control whether or not to output a signal from the signal processing circuit SP while controlling the activation / inactivation of the avalanche diode D. In addition, all of the above-mentioned embodiments are merely examples of concrete implementations of the present invention, and the technical scope of the present invention should not be interpreted in a limited manner based on these examples. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]
[0083] 10 Pixel Area 11 Photoelectric conversion element 14 Selection methods 15 Switching Method D Avalanche diode Vdd voltage Vss Voltage VH Voltage VL Voltage
Claims
1. a plurality of photoelectric conversion elements each including an avalanche diode and a counter that counts a signal based on light incident on the avalanche diode; a first selection means is provided corresponding to a first photoelectric conversion element among the plurality of photoelectric conversion elements; a second selection means is provided corresponding to a second photoelectric conversion element among the plurality of photoelectric conversion elements; In a first mode, the first selection means is controlled to a state in which a signal from the first photoelectric conversion element is read out, and the second selection means is controlled to a state in which a signal from the second photoelectric conversion element is not read out, An imaging device characterized in that, in a second mode which is set during a period different from the period during which the first mode is set, the second photoelectric conversion element which has been controlled to a state in which the signal is not read out is controlled by the second selection means to a state in which the signal is read out, and the first photoelectric conversion element which has been controlled to a state in which the signal is read out is controlled by the first selection means to a state in which the signal is not read out.
2. 2. The imaging device according to claim 1, wherein, in the first mode, the first selection means controls the avalanche diode of the first photoelectric conversion element to be in a state in which a reverse bias voltage capable of avalanche multiplication is applied thereto, thereby causing a signal to be read out from the first photoelectric conversion element, and the second selection means controls the avalanche diode of the second photoelectric conversion element to be in a state in which a reverse bias voltage capable of avalanche multiplication is not applied thereto, thereby causing a signal to be read out from the second photoelectric conversion element.
3. At least some of the plurality of photoelectric conversion elements are arranged in a first direction, the first mode is a period during which a first vertical scan is being performed, 3. The imaging device according to claim 1, wherein the second mode is a period during which a second vertical scan is performed.
4. A vertical scanning circuit and a horizontal scanning circuit are provided, 4. The image pickup apparatus according to claim 1, wherein the first selection means and the second selection means are controlled by the vertical scanning circuit and the horizontal scanning circuit.
5. each of the plurality of photoelectric conversion elements includes a quench element connected to the avalanche diode; 5. The imaging device according to claim 1, wherein the first selection means controls the on / off of the quench element of the first photoelectric conversion element.
6. Each of the plurality of photoelectric conversion elements is connected to a control line that controls a potential applied to an anode or a cathode of the avalanche diode; the control line includes a first voltage line that supplies a potential that causes the avalanche diode to be in a state in which avalanche multiplication does not occur, and a second voltage line that supplies a potential that causes the avalanche diode to be in a state in which avalanche multiplication is possible, 5. The imaging device according to claim 1, wherein the first selection means and the second selection means select a control line to be connected to the avalanche diode from the first voltage line and the second voltage line.
7. 5. The imaging device according to claim 1, wherein the first selection means controls whether or not to read a signal from the counter of the first photoelectric conversion element, and the second selection means controls whether or not to read a signal from the counter of the second photoelectric conversion element.
8. 5. The imaging apparatus according to claim 1, wherein in the first mode, the counter included in the second photoelectric conversion element is controlled by the second selection means so as not to perform counting.
9. 9. The image pickup apparatus according to claim 1, wherein the first selection means is constituted by a combinational circuit.
10. 10. The imaging device according to claim 9, wherein the combinational circuit is an AND circuit.
11. 11. The imaging device according to claim 1, wherein the first mode and the second mode have different signal readout areas.
12. 12. The imaging device according to claim 1, wherein in the first mode, photoelectric conversion is performed between the avalanche diode of the first photoelectric conversion element and the avalanche diode of the second photoelectric conversion element.
13. 13. The imaging device according to claim 1, wherein in the second mode, the first selection means, which is controlled to a state in which the signal is read out, is maintained in a state in which the signal is read out by the first selection means.
14. When an object is captured in an area where the image is captured by the image capture device, 14. The imaging device according to claim 13, wherein the first mode is changed to the second mode.
15. A first substrate and a second substrate laminated on the first substrate, the first substrate includes the counter, 15. The imaging device according to claim 1, wherein the second substrate includes the avalanche diode.
16. The photoelectric conversion element includes a waveform shaping unit, The imaging device according to claim 15 , wherein the first substrate includes the waveform shaping section.
17. The photoelectric conversion element includes a quench element, The imaging device according to claim 16 , wherein the first substrate includes the quenching element.
18. An imaging device according to any one of claims 1 to 17; a signal processing unit that processes a signal output from the imaging device.
19. An imaging device according to any one of claims 1 to 17; a distance information acquisition means for acquiring distance information to an object from information based on a signal from the imaging device, A moving body further comprising a control means for controlling the moving body based on the distance information.
Citation Information
Patent Citations
Stacked chip SPAD image sensor
US20150115131A1
