Photoelectric conversion device, light detection system, movable body, and method of controlling photoelectric conversion device
The photoelectric conversion device simplifies its circuit configuration by using a switch unit and periodic signals with phase differences to achieve multiple photon detection periods, allowing for varied sensitivities and reducing power consumption.
Patent Information
- Application Number
- JP2023075577
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
AI Technical Summary
The method described in Patent Document 1 requires gating signals with different pulse widths for each counter, complicating the circuit configuration of photoelectric conversion devices.
A photoelectric conversion device with a switch unit that changes voltage to ground and blocks current flow, generating detection signals based on voltage changes, using periodic signals with predetermined phase differences to simplify the circuit configuration and allow for multiple photon detection periods without requiring different pulse widths.
The circuit configuration is simplified, enabling the device to count photons with various sensitivities without complicating the circuit, expanding the dynamic range and reducing power consumption.
Smart Images

Figure 2024160505000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a photoelectric conversion device, a light detection system, a moving object, and a method for controlling a photoelectric conversion device. [Background technology]
[0002] Patent Document 1 discloses a method of inputting an output signal from one single photon avalanche diode (SPAD) to two counters and gating the SPAD output signal input to each counter. In Patent Document 1, the photon detection sensitivity of each counter is adjusted by the period during which the gating signal is High (the pulse width of the gating signal). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2022-96472 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the method described in Patent Document 1 requires gating signals with different pulse widths for each counter, which can complicate the circuit configuration of the photoelectric conversion device.
[0005] The present disclosure has been made in consideration of the above-mentioned problems, and has an object to simplify the circuit configuration of a photoelectric conversion device. [Means for solving the problem]
[0006] A photoelectric conversion device according to one embodiment of the present disclosure includes a detection unit that generates a voltage change in response to incidence of photons, a switch unit that is provided between the detection unit and a power supply node and that changes the voltage of the detection unit to a predetermined bias voltage by supplying a current from the power supply node to the detection unit in an on state and cuts off the current from the power supply node to the detection unit in an off state, a first circuit unit that generates a first detection signal based on the voltage change at the detection unit and a first periodic signal, a second circuit unit that generates a second detection signal based on the voltage change at the detection unit and a second periodic signal having a predetermined phase difference with respect to the first periodic signal, a counter unit that counts the first detection signal or the second detection signal and outputs a count value, and a control unit that generates a first signal for causing the first circuit unit to generate the first detection signal and a second signal for causing the second circuit unit to generate the second detection signal, wherein the first periodic signal and the second periodic signal are generated based on the first signal or the second signal. Effect of the Invention
[0007] According to the present disclosure, the circuit configuration of a photoelectric conversion device can be simplified. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a block diagram of an imaging system according to a first embodiment. [Diagram 2] 1 is a block diagram of a photoelectric conversion device according to a first embodiment. [Diagram 3] FIG. 2 is a circuit diagram of a pixel unit according to the first embodiment. [Figure 4] 4 is a timing chart of signals input to a pixel unit in the first embodiment. FIG. [Diagram 5] 5 is a timing chart showing the operation of a counting unit in the first embodiment. FIG. [Figure 6] FIG. 11 is a circuit diagram of a pixel unit according to a second embodiment. [Figure 7A] 10 is a graph showing an output of a count value output by a counting section in the second embodiment. [Figure 7B] 10 is a graph showing an output of a count value output by a counting section in the second embodiment. [Figure 8] FIG. 13 is a circuit diagram of a pixel unit according to a third embodiment. [Figure 9] FIG. 11 is a timing chart showing the operation of a counting unit in the third embodiment. [Figure 10] FIG. 13 is a schematic diagram of a photoelectric conversion device according to a fourth embodiment. [Figure 11] 13A and 13B are diagrams illustrating an example of an arrangement of a sensor substrate in a fourth embodiment. [Figure 12] 13A and 13B are diagrams illustrating an example of an arrangement of circuit boards in a fourth embodiment. [Figure 13] FIG. 13 is a circuit diagram of an APD and a pulse generating unit in a fourth embodiment. [Figure 14] 13 is a diagram illustrating the relationship between the operation of an APD and an output signal in the fourth embodiment. [Figure 15] FIG. 13 is a block diagram of a light detection system according to a fifth embodiment. [Figure 16] FIG. 13 is a schematic diagram of an endoscopic surgery system according to a sixth embodiment. [Figure 17A] FIG. 13 is a schematic diagram of a light detection system according to a seventh embodiment. [Figure 17B] FIG. 13 is a schematic diagram of a moving body in a seventh embodiment. [Figure 18] 20 is a flowchart showing the operation of the light detection system in the seventh embodiment. [Figure 19] FIG. 13 is a diagram showing a specific example of an electronic device in the eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Elements having common functions throughout the drawings will be designated by the same reference numerals, and duplicated descriptions may be omitted or simplified.
[0010] [First embodiment] FIG. 1 is a block diagram of an imaging system 110 in the first embodiment. The imaging system 110 is one form of a light detection system. The imaging system 110 includes a barrier 111, a lens 112, an aperture 113, a photoelectric conversion device 100, a signal processing unit 114, a timing generation unit 115, an overall control / calculation unit 116, a memory unit 117, a recording medium control I / F unit 118, a recording medium 119, and an external I / F unit 120. The barrier 111 protects the lens, and the lens 112 forms an optical image of a subject on the photoelectric conversion device 100. The aperture 113 changes the amount of light that passes through the lens 112. The photoelectric conversion device 100 converts the optical image formed by the lens 112 into image data. The signal processing unit 114 performs various corrections and data compression on the imaging data output from the photoelectric conversion device 100. For example, the signal processing unit 114 performs pixel interpolation processing, color conversion processing, and the like, based on the pixel signals output from the photoelectric conversion device 100. Furthermore, the signal processing unit 114 generates digital image data in RGB format, YUV format, and the like, based on the pixel signals.
[0011] The timing generating unit 115 outputs various timing signals to the photoelectric conversion device 100 and the signal processing unit 114. The overall control and calculation unit 116 may include an image signal processor (ISP), an application specific integrated circuit (ASIC), a programmable gate array (FPGA), or the like. The overall control and calculation unit 116 controls the entire imaging system 110. The memory unit 117 temporarily stores image data. The recording medium control I / F unit 118 is an interface for recording or reading image data in a recording medium 119, and the recording medium 119 is a removable recording medium such as a semiconductor memory for recording or reading imaging data. The external I / F unit 120 is an interface for communicating with an external computer, etc. Timing signals, etc. may be input from outside the imaging system 110, and the imaging system 110 only needs to have at least the photoelectric conversion device 100 and a signal processing unit 114 for processing an image signal output from the photoelectric conversion device 100.
[0012] In this embodiment, the photoelectric conversion device 100 and the signal processing unit 114 are provided on different semiconductor substrates, but the photoelectric conversion device 100 and the signal processing unit 114 may be formed on the same semiconductor substrate. The imaging system 110 is a light detection system, and examples of the imaging system 110 include a digital still camera, a digital camcorder, a camera head, a copier, a fax machine, a mobile phone, an in-vehicle camera, an observation satellite, and a surveillance camera.
[0013] Each pixel may include a first photoelectric conversion unit and a second photoelectric conversion unit. The signal processing unit 114 processes a pixel signal based on the charge generated in the first photoelectric conversion unit and a pixel signal based on the charge generated in the second photoelectric conversion unit, and may acquire distance information from the imaging system 110 to the subject.
[0014] 2 is a block diagram of a photoelectric conversion device 100 according to the first embodiment. The photoelectric conversion device 100 includes a pixel section 50, a drive control section 60, a vertical scanning circuit 31, a horizontal scanning circuit 32, a register block 40, a digital processing section 70, an output section 80, and delay sections 91 and 92.
[0015] The pixel unit 50 outputs a pixel signal according to the incident light via the output line 16. The pixel units 50 are arranged in a matrix in the pixel region of the photoelectric conversion device 100. The number of pixel units 50 is not particularly limited. For example, when used in an imaging camera, 1920 pixel units 50 may be arranged in a matrix in the horizontal direction and 1080 pixel units 50 in the vertical direction. When used in a line scan sensor, the pixel units 50 may be arranged in one row and multiple columns. A detailed configuration of the pixel unit 50 will be described later.
[0016] The drive control unit 60 receives a timing signal S5 from the overall control and calculation unit 116 via the timing generation unit 115. The drive control unit 60 transmits a drive signal to the vertical scanning circuit 31 and the horizontal scanning circuit 32 based on the timing signal. The vertical scanning circuit 31 and the horizontal scanning circuit 32 are controlled by the drive signal. The drive control unit 60 also transmits a periodic signal S1 to the pixel unit 50 via the periodic signal line 15 based on the timing signal. The SPAD 510 of the pixel unit 50 is charged based on the periodic signal S1. The SPAD 510 will be described later. The drive control unit 60 supplies a periodic signal S2 synchronized with the periodic signal S1 to the delay unit 91 and the delay unit 92. The drive control unit 60 also supplies other control signals (not shown) to control the overall operation of the photoelectric conversion device 100. The drive control unit 60 may be driven using a clock from a phase-locked loop (PLL) or the like inside the photoelectric conversion device 100.
[0017] The vertical scanning circuit 31 receives a drive signal from the drive control unit 60 and supplies a scan signal to each row of the pixel unit 50 via the scanning line 26. The vertical scanning circuit 31 sequentially scans the pixel unit 50 row by row. That is, the vertical scanning circuit 31 selects a pixel row to output a pixel signal. The vertical scanning circuit 31 may be composed of logic circuits such as a shift register and an address decoder.
[0018] The horizontal scanning circuit 32 receives a driving signal from the driving control unit 60 and supplies a scanning signal to the pixel units 50 of each column via a signal line 29. The horizontal scanning circuit 32 sequentially scans the pixel units 50 column by column. That is, the horizontal scanning circuit 32 selects a pixel column to output a pixel signal. The horizontal scanning circuit 32 may be configured with logic circuits such as a shift register and an address decoder. A pixel signal corresponding to the incident light is output from the pixel units 50 selected by the vertical scanning circuit 31 and the horizontal scanning circuit 32 via an output line 16.
[0019] The register block 40 receives a setting signal S6 for setting the photon detection sensitivity in the pixel section 50 from the overall control and calculation section 116 via the timing generation section 115. The register block 40 supplies a selection signal S4 for selecting the photon detection sensitivity (detection period) to the pixel section 50 via the setting signal line 12 based on the setting signal S6. The register block 40 also supplies a control signal Sd1 to the delay section 91 and a control signal Sd2 to the delay section 92. The control signals Sd1 and Sd2 will be described later. Furthermore, the register block 40 can hold various setting parameters for the photodetection operation. For example, the register block 40 can store and transmit the setting of the timing of the photodetection operation, the conditions for driving the pixels, the setting of the driving capacity in the analog driving section in the photoelectric conversion device, and the like. As data communication for transmitting the setting information, for example, a serial communication means such as I2C (Inter-Integrated Circuit) or SPI (Serial Peripheral Interface) can be used, but another communication means may also be used.
[0020] The delay unit 91 receives a periodic signal S2 (first signal) from the drive control unit 60, and receives a control signal Sd1 from the register block 40. The control signal Sd1 includes a parameter for setting a delay time for the input periodic signal S2. The delay unit 91 generates a periodic signal Sg1 (first periodic signal) by delaying the periodic signal S2 by a predetermined time based on the control signal Sd1. The delay unit 91 supplies the generated periodic signal Sg1 to the pixel unit 50. The periodic signal Sg1 will be described later.
[0021] The delay unit 92 receives a periodic signal S2 (second signal) from the drive control unit 60, and receives a control signal Sd2 from the register block 40. The control signal Sd2 includes a parameter for setting a delay time for the input periodic signal S2. The delay time set by the control signal Sd2 is different from the delay time set by the control signal Sd1. The delay unit 92 generates a periodic signal Sg2 (second periodic signal) by delaying the periodic signal S2 by a predetermined time based on the control signal Sd2. The delay unit 92 supplies the generated periodic signal Sg2 to the pixel unit 50. The periodic signal Sg2 will be described later.
[0022] The digital processing unit 70 receives a timing signal S7 from the overall control / arithmetic unit 116 via the timing generation unit 115. The digital processing unit 70 also receives a pixel signal from the pixel unit 50 via the output line 16. The digital processing unit 70 performs various digital signal processing such as offset processing, digital gain processing, etc. on the pixel signal based on the timing signal S7.
[0023] The output unit 80 receives the processed pixel signal from the digital processing unit 70. The output unit 80 transmits the pixel signal received from the digital processing unit 70 as an output signal S10 to the overall control / calculation unit 116 via the signal processing unit 114. The output unit 80 may include a serializer that converts parallel data into serial data, an LVDS (Low Voltage Differential Signal) transmitter, etc. The output unit 80 may include parallel output means, etc. instead of a serializer.
[0024] 3 is a circuit diagram of a pixel unit 50 in the first embodiment. The pixel unit 50 includes a SPAD 510, a switch unit 520, a circuit unit 530 (first circuit unit), a circuit unit 540 (second circuit unit), a counter unit 551, a counter unit 552, and a selection unit 560.
[0025] The SPAD 510 is a photodiode that functions as a detection unit that detects incidence of photons on the pixel unit 50. The cathode of the SPAD 510 is connected to one end of the NOT gate 581 via a node N1. The anode of the SPAD 510 is connected to a node of a voltage source that supplies a voltage VL. The operation of the SPAD 510 will be described later. Note that, although the SPAD 510 will be described in this embodiment as being a single-photon avalanche diode, an avalanche photodiode (APD), which will be described later, may also be used.
[0026] The switch unit 520 connects and disconnects the node (power supply node) of the voltage source that supplies the voltage VH and the SPAD 510. The switch unit 520 according to this embodiment is composed of a MOS transistor. The gate of the switch unit 520 is connected to the periodic signal line 15. The source of the switch unit 520 is connected to the node of the voltage source that supplies the voltage VH. The drain of the switch unit 520 is connected to the cathode of the SPAD 510 and one end of the NOT gate 581 via the node N1. A periodic signal S1 is supplied to the gate of the switch unit 520 from the drive control unit 60 via the periodic signal line 15. The periodic signal S1 may be a square wave (pulse signal) having a predetermined amplitude and pulse width. While the periodic signal S1 is at a low level, the switch unit 520 is in an on state, the voltage source and the SPAD 510 are electrically connected, and a current is supplied from the voltage source to the SPAD 510 via the switch unit 520. Therefore, while the periodic signal S1 is at a low level, the SPAD 510 is charged, and the potential of the node N1 rises until it reaches a predetermined bias voltage. While the periodic signal S1 is at a high level, the switch unit 520 is in an off state, the voltage source and the SPAD 510 are electrically disconnected, and the supply of current from the voltage source to the SPAD 510 is cut off. Therefore, while the periodic signal S1 is at a high level, the SPAD 510 is not charged, and the potential of the node N1 is maintained at the potential when the switch unit 520 is switched from the on state to the off state. The voltage source that supplies the voltages VH and VL supplies a reverse bias voltage required for the SPAD 510 to perform an avalanche multiplication operation via the switch unit 520. When a photon is incident on the SPAD 510, avalanche multiplication occurs in the SPAD 510, and the potential of the node N1 drops. That is, a current generated in response to the input of a photon to the SPAD 510 causes a change in the potential of the node N1 (the voltage of the SPAD 510). Depending on the potential of node N1, NOT gate 581 transmits inverted signal S3 to node N2. When the potential of node N1 becomes less than the logic threshold due to incidence of a photon on SPAD 510, NOT gate 581 transmits inverted signal S3 of high level. On the other hand, when the potential of node N1 becomes equal to or greater than the logic threshold due to charging of SPAD 510, NOT gate 581 transmits inverted signal S3 of low level.
[0027] The circuit unit 530 outputs a detection signal S8 (first detection signal) indicating the detection of a photon in the SPAD 510 based on the inverted signal S3 via node N2 and the periodic signal Sg1 from the delay unit 91. The circuit unit 530 includes an AND gate 531. One input node of the AND gate 531 is connected to the node N2, and the inverted signal S3 is input to the AND gate 531. The other input node of the AND gate 531 is connected to the delay unit 91, and the periodic signal Sg1 is input to the AND gate 531. The circuit unit 530 performs a logical AND operation on the signal input to the AND gate 531. The circuit unit 530 outputs the result of the operation based on the inverted signal S3 and the periodic signal Sg1 to the counter unit 551 as a detection signal S8.
[0028] The circuit unit 540 outputs a detection signal S9 (second detection signal) indicating the detection of a photon in the SPAD 510 based on the inverted signal S3 via the node N2 and the periodic signal Sg2 from the delay unit 92. The circuit unit 540 includes an AND gate 541. One input node of the AND gate 541 is connected to the node N2, and the inverted signal S3 is input to the AND gate 541. That is, the inverted signal S3 is commonly input to the circuit units 530 and 540. The other input node of the AND gate 541 is connected to the delay unit 92, and the periodic signal Sg2 is input to the AND gate 541. The circuit unit 540 performs a logical AND operation on the signal input to the AND gate 541. The circuit unit 540 outputs the result of the operation based on the inverted signal S3 and the periodic signal Sg2 to the counter unit 552 as the detection signal S9.
[0029] The counters 551 and 552 count the detection signal indicating the detection of the incidence of photons on the SPAD 510, and output the count result as a count value. Specifically, the counter 551 counts the number of times that the detection signal S8 transmitted from the circuit unit 530 transitions from a low level to a high level as a count value C1 for a predetermined period (for example, one frame period). The counter 552 counts the number of times that the detection signal S9 transmitted from the circuit unit 540 transitions from a low level to a high level as a count value C2 for a predetermined period (for example, one frame period). After the predetermined period (for example, one frame period) has elapsed, the counters 551 and 552 output the count values C1 and C2 to the selection unit 560, respectively, as pixel values. The counters 551 and 552 initialize the count values C1 and C2 after outputting the count values C1 and C2 to the selection unit 560, respectively.
[0030] The selection unit 560 receives a plurality of count values including the count values C1 and C2 from the count units 551 and 552. The selection unit 560 selects one count value from the plurality of count values and outputs the selected count value to the digital processing unit 70 via the output line 16. Specifically, the selection unit 560 receives a selection signal S4 from the register block 40. The selection signal S4 includes information specifying the count value selected by the selection unit 560. The selection unit 560 selects the count value C1 received from the count unit 551 or the count value C2 received from the count unit 552 based on the selection signal S4. When the pixel unit 50 is selected based on the scanning signal from the vertical scanning circuit 31 and the scanning signal from the horizontal scanning circuit 32, the selection unit 560 outputs the count value selected based on the selection signal S4 to the digital processing unit 70 as a pixel value.
[0031] The selection signal S4 may be a 1-bit signal. For example, when the selection signal S4 is at a low level, the selection unit 560 outputs the count value C1 output by the counting unit 551. Also, for example, when the selection signal S4 is at a high level, the selection unit 560 outputs the count value C2 output by the counting unit 552. The selection signal S4 may be a 2-bit or more signal. Specifically, the number of bits of the selection signal S4 may be determined according to the number of counting units that count the count value. Also, the selection unit 560 may read only the count value specified by the selection signal S4, or may read all the count values input to the selection unit 560. That is, only the count values output from some of the multiple counting units may be read by the selection unit 560, or the count values output from all of the multiple counting units may be read by the selection unit 560.
[0032] Fig. 4 is a timing diagram of signals input to the pixel unit 50 in the first embodiment. Fig. 4 shows timing diagrams of a periodic signal S1 input to the pixel unit 50, a periodic signal S2 input commonly to the delay units 91 and 92, a periodic signal Sg1 output from the delay unit 91, and a periodic signal Sg2 output from the delay unit 92.
[0033] The periodic signal S1 determines the timing for charging the SPAD 510 of the pixel unit 50. In this embodiment, the SPAD 510 is charged when the periodic signal S1 is at a low level, and is not charged when the periodic signal S1 is at a high level. The periodic signal S2 is synchronized with the periodic signal S1, and is a pulse signal that rises at the time when the periodic signal S1 falls. In this embodiment, the frequency of the periodic signal S2 is the same as the frequency of the periodic signal S1.
[0034] The periodic signal Sg1 is a pulse signal and has the same frequency and pulse width as the periodic signal S2. On the other hand, the periodic signal Sg1 is delayed by a period Td1 with respect to the periodic signal S2. That is, there is a phase difference (first phase difference) corresponding to the period Td1 between the periodic signal Sg1 and the periodic signal S2. The period Td1 is determined based on the control signal Sd1 transmitted from the register block 40 to the delay unit 91.
[0035] The periodic signal Sg2 is a pulse signal and has the same frequency and pulse width as the periodic signal S2. Meanwhile, the periodic signal Sg2 is delayed by a period Td2 with respect to the periodic signal S2. That is, there is a phase difference (second phase difference) between the periodic signal Sg2 and the periodic signal S2 corresponding to the period Td2. The period Td2 is shorter than the period Td1. Therefore, the phase difference between the periodic signal Sg2 and the periodic signal S2 is smaller than the phase difference between the periodic signal Sg1 and the periodic signal S2. That is, the periodic signal Sg2 has a phase difference with respect to the periodic signal Sg1 corresponding to a period (Td2-Td1) obtained by subtracting the period Td1 from the period Td2. The period Td2 is determined based on the control signal Sd2 transmitted from the register block 40 to the delay unit 92.
[0036] The length of the photon detection period by the SPAD 510 of the pixel unit 50 corresponds to the photon detection sensitivity of the SPAD 510. The length of the photon detection period by the SPAD 510 changes according to the periods Td1 and Td2. That is, the photon detection sensitivity by the SPAD 510 depends on the phase difference between the periodic signal S2 and the periodic signals Sg1 and Sg2. Two periodic signals Sg1 and Sg2, which are generated based on the common periodic signal S2 and have different phase differences with respect to the periodic signal S2, are input to the pixel unit 50. That is, the SPAD 510 of the pixel unit 50 can operate to have a plurality of detection sensitivities based on the common periodic signal S2.
[0037] 5 is a timing diagram showing the operation of the counting unit 551 and the counting unit 552 in the first embodiment. Each of the periods P1 to P6 corresponds to one period of the periodic signal S1. FIG. 5 shows timing diagrams of the timing Tin of the incidence of photons to the SPAD 510 in the periods P1 to P6, the periodic signal S1 input to the gate of the switch unit 520, the potential Pn of the node N1, the inverted signal S3 input to the node N2, the detection period Pd1 of photons via the circuit unit 530, the periodic signal Sg1 from the delay unit 91, the detection signal S8 output from the circuit unit 530, the count value C1 output from the counting unit 551, the detection period Pd2 of photons via the circuit unit 540, the periodic signal Sg2 from the delay unit 92, the detection signal S9 output from the circuit unit 540, and the count value C2 output from the counting unit 552.
[0038] At time t0 before the period P1, the periodic signal S1 is at a high level, but the periodic signals Sg1 and Sg2 are at a low level. Therefore, at time t0, the detection signals S8 and S9 are at a low level. Also, at time t0, the potential Pn is less than the logic threshold Th, and the inversion signal S3 is at a high level. Also, at time t0, the count value C1 is n, and the count value C2 is m.
[0039] Since the periodic signal S1 is at a high level from time t0 to the start time t1 of the period P1, the switch section 520 is maintained in an off state. Therefore, in the period from time t0 to t1, the SPAD 510 is not charged and the potential Pn is maintained below the logic threshold Th. Therefore, in the period from time t0 to t1, the inverted signal S3 is maintained at a high level. On the other hand, in the period from time t0 to t1, the periodic signals Sg1 and Sg2 are at a low level, so the detection signals S8 and S9 are maintained at a low level.
[0040] At time t1, the periodic signal S1 transitions from a high level to a low level, and the switch unit 520 switches from an off state to an on state. When the switch unit 520 turns on, charging of the SPAD 510 starts, and the detection period Pd1 and the detection period Pd2 in the period P1 start at the same time. Since the periodic signal S1 is maintained at a low level during the period from time t1 to t2, the switch unit 520 is maintained in an on state. Therefore, the SPAD 510 is charged during the period from time t1 to t2, but the potential Pn does not exceed the logic threshold Th during the period from time t1 to t2. Therefore, the inversion signal S3 is maintained at a high level during the period from time t1 to t2. Meanwhile, since the periodic signals Sg1 and Sg2 are at a low level during the period from time t1 to t2, the detection signals S8 and S9 are maintained at a low level.
[0041] At time t2, the potential Pn reaches the logic threshold Th, and the inversion signal S3 transitions from high to low. Since the periodic signal S1 is maintained at a low level during the period from time t2 to t3, the switch unit 520 is maintained in an on state. That is, since the charging of the SPAD 510 continues, the potential Pn rises above the logic threshold Th during the period from time t2 to t3. Therefore, the inversion signal S3 is maintained at a low level during the period from t2 to t3. Furthermore, the periodic signals Sg1 and Sg2 are at a low level during the period from time t2 to t3. Therefore, the detection signals S8 and S9 are maintained at a low level during the period from time t2 to t3.
[0042] At time t3, the periodic signal S1 transitions from a low level to a high level, and the switch unit 520 switches from an on state to an off state. When the switch unit 520 switches to an off state, charging of the SPAD 510 stops. That is, the rise in the potential Pn stops. Since the periodic signal S1 is maintained at a high level during the period from time t3 to t4, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t3 to t4, the potential Pn is maintained at the potential at time t3, and the inverted signal S3 is maintained at a low level. Furthermore, the periodic signals Sg1 and Sg2 are at a low level during the period from time t3 to t4. Therefore, the detection signals S8 and S9 are maintained at a low level during the period from time t3 to t4.
[0043] At time t4, the periodic signal Sg2 transitions from a low level to a high level, and is maintained at a high level during the period from time t4 to t5. Furthermore, since the periodic signal S1 is maintained at a high level during the period from time t4 to t5, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t4 to t5, the potential Pn is maintained at the potential at time t4, and the inverted signal S3 is maintained at a low level. Furthermore, the periodic signal Sg1 is at a low level during the period from time t4 to t5. Therefore, the detection signal S8 and the detection signal S9 are maintained at a low level during the period from time t4 to t5.
[0044] At time t5, the periodic signal Sg2 transitions from high level to low level, and the detection period Pd2 in the period P1 ends. The periodic signal Sg2 is maintained at low level during the period from time t5 to t6. Furthermore, since the periodic signal S1 is maintained at high level during the period from time t5 to t6, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t5 to t6, the potential Pn is maintained at the potential at time t5, and the inversion signal S3 is maintained at low level. Furthermore, the periodic signal Sg1 is at low level during the period from time t5 to t6. Therefore, the detection signal S8 and the detection signal S9 are maintained at low level during the period from time t5 to t6.
[0045] At time t6, the periodic signal Sg1 transitions from a low level to a high level, and is maintained at a high level during the period from time t6 to t7. Furthermore, since the periodic signal S1 is maintained at a high level during the period from time t6 to t7, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t6 to t7, the potential Pn is maintained at the potential at time t6, and the inverted signal S3 is maintained at a low level. Furthermore, the periodic signal Sg2 is at a low level during the period from time t6 to t7. Therefore, the detection signals S8 and S9 are maintained at a low level during the period from time t6 to t7.
[0046] At time t7, the periodic signal Sg1 transitions from high level to low level, and the detection period Pd1 in the period P1 ends. The periodic signal Sg1 is maintained at low level during the period from time t7 to t8. Furthermore, since the periodic signal S1 is maintained at high level during the period from time t7 to t8, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t7 to t8, the potential Pn is maintained at the potential at time t7, and the inversion signal S3 is maintained at low level. Furthermore, the periodic signal Sg2 is at low level during the period from time t7 to t8. Therefore, the detection signal S8 and the detection signal S9 are maintained at low level during the period from time t7 to t8.
[0047] At time t8, period P1 ends and period P2 starts at the same time. In the following description, the same operations of each component or changes in each signal as those already described may be omitted or simplified.
[0048] The operation of each component and the change of each signal at time t8 are the same as those at time t1. In the period from time t8 to t9, the switch section 520 is in an on state. Therefore, the SPAD 510 can be charged, but the SPAD 510 does not perform avalanche multiplication in the immediately preceding period P1. Therefore, the potential Pn is maintained at the potential at time t8 in the period from time t8 to t9. That is, the potential Pn is maintained at or above the logic threshold Th in the period from time t8 to t9, and the inversion signal S3 is maintained at a low level. Therefore, the detection signal S8 and the detection signal S9 are maintained at a low level in the period from time t8 to t9.
[0049] The operation of each component and the change of each signal during the period from time t9 to t10 are the same as those during the period from time t3 to t4. At time t10, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logic threshold Th, and the inversion signal S3 transitions from low level to high level. Since the periodic signal S1 is maintained at a high level during the period from time t10 to t11, the switch unit 520 is maintained in an off state. Therefore, during the period from time t10 to t11, the potential Pn is maintained at a potential less than the logic threshold Th at time t10, and the inversion signal S3 is maintained at a high level. On the other hand, the periodic signals Sg1 and Sg2 are at a low level during the period from time t10 to t11. Therefore, in the period from time t10 to time t11, the detection signal S8 and the detection signal S9 are maintained at a low level.
[0050] At time t11, the periodic signal Sg2 transitions from a low level to a high level. Also, at time t11, the inverted signal S3 is at a high level. Therefore, the detection signal S9 transitions from a low level to a high level at time t11. In response to the transition of the detection signal S9 to a high level, the counting unit 552 executes an increment process. That is, the counting unit 552 updates the count value from m to m+1 at time t11. On the other hand, since the periodic signal Sg1 is at a low level at time t11, the detection signal S8 is at a low level at time t11. Therefore, the counting unit 551 does not execute an increment process at time t11, and the count value held by the counting unit 551 is maintained at n. During the period from time t11 to t12, the periodic signal Sg2 is maintained at a high level. Also, since the periodic signal S1 is maintained at a high level during the period from time t11 to t12, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t11 to t12, the potential Pn is maintained at the potential at time t11, and the inverted signal S3 is maintained at a high level. Therefore, the detection signal S9 is maintained at a high level. Meanwhile, the periodic signal Sg1 is maintained at a low level during the period from time t11 to t12. Therefore, the detection signal S8 is maintained at a low level during the period from time t11 to t12.
[0051] At time t12, the periodic signal Sg2 transitions from high level to low level. In response to the transition of the periodic signal Sg2 to low level, the detection signal S9 transitions from high level to low level at time t12. Since the periodic signal S1 is maintained at high level during the period from time t12 to t13, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t12 to t13, the potential Pn is maintained at the potential at time t12, and the inversion signal S3 is maintained at high level. On the other hand, the periodic signals Sg1 and Sg2 are maintained at low level during the period from time t12 to t13. Therefore, the detection signals S8 and S9 are maintained at low level during the period from time t12 to t13.
[0052] At time t13, the periodic signal Sg1 transitions from a low level to a high level. Also, at time t13, the inverted signal S3 is at a high level. Therefore, the detection signal S8 transitions from a low level to a high level at time t13. In response to the transition of the detection signal S8 to a high level, the counting unit 551 performs an increment process. That is, the counting unit 551 updates the count value from n to n+1 at time t13. On the other hand, since the periodic signal Sg2 is at a low level at time t13, the detection signal S9 is at a low level at time t13. Therefore, the counting unit 552 does not perform an increment process at time t13, and the count value held by the counting unit 552 is maintained at m+1. During the period from time t13 to t14, the periodic signal Sg1 is maintained at a high level. Also, since the periodic signal S1 is maintained at a high level during the period from time t13 to t14, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t13 to t14, the potential Pn is maintained at the potential at time t13, and the inverted signal S3 is maintained at a high level. Therefore, the detection signal S8 is maintained at a high level. Meanwhile, the periodic signal Sg2 is maintained at a low level during the period from time t13 to t14. Therefore, the detection signal S9 is maintained at a low level during the period from time t13 to t14.
[0053] At time t14, the periodic signal Sg1 transitions from high level to low level. In response to the transition of the periodic signal Sg1 to low level, the detection signal S8 transitions from high level to low level at time t14. Since the periodic signal S1 is maintained at high level during the period from time t14 to t15, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t14 to t15, the potential Pn is maintained at the potential at time t14, and the inversion signal S3 is maintained at high level. On the other hand, the periodic signals Sg1 and Sg2 are maintained at low level during the period from time t14 to t15. Therefore, the detection signals S8 and S9 are maintained at low level during the period from time t14 to t15.
[0054] At time t15, period P2 ends and period P3 starts at the same time. In the following description, the same operations or signal changes as those in periods P1 and P2 may be omitted or simplified.
[0055] The operation and changes of each signal during the period from time t15 to t20 are the same as those during the period from time t1 to t6. At time t20, the periodic signal Sg1 transitions from a low level to a high level, and is maintained at a high level during the period from time t20 to t21. Furthermore, since the periodic signal S1 is maintained at a high level during the period from time t20 to t21, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t20 to t21, the potential Pn is maintained at the potential at time t20, and the inverted signal S3 is maintained at a low level. Furthermore, the periodic signal Sg2 is at a low level during the period from time t20 to t21. Therefore, the detection signal S8 and the detection signal S9 are maintained at a low level during the period from time t20 to t21.
[0056] At time t21, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. The periodic signal Sg1 is at high level at time t21. Therefore, the detection signal S8 transitions from low level to high level at time t21. In response to the transition of the detection signal S8 to high level, an increment process is performed in the counter 551. That is, the counter 551 updates the count value from n+1 to n+2 at time t21. Meanwhile, since the periodic signal Sg2 is at low level at time t21, the detection signal S9 is at low level at time t21. Therefore, at time t21, the counting unit 552 does not perform an increment process, and the count value held by the counting unit 552 is maintained at m+1. During the period from time t21 to t22, the periodic signal S1 is maintained at a high level, and the switch unit 520 is maintained in an off state. Therefore, during the period from time t21 to t22, the potential Pn is maintained at a potential less than the logical threshold value Th at time t21, and the inversion signal S3 is maintained at a high level. Furthermore, the periodic signal Sg1 is at a high level during the period from time t21 to t22. Therefore, during the period from time t21 to t22, the detection signal S8 is maintained at a high level. Meanwhile, during the period from time t21 to t22, the periodic signal Sg2 is at a low level. Therefore, during the period from time t21 to t22, the detection signal S9 is maintained at a low level. The operation of each component and the changes in each signal during the period from time t22 to t23 are the same as the operation of each component and the changes in each signal during the period from time t14 to t15.
[0057] At time t23, period P3 ends and period P4 starts at the same time. The operation of each component and the change in each signal in the period from time t23 to t29 are the same as the operation of each component and the change in each signal in the period from time t1 to t7.
[0058] At time t29, the periodic signal Sg1 transitions from high level to low level, and is maintained at low level during the period from time t29 to t30. Furthermore, since the periodic signal S1 is maintained at high level during the period from time t29 to t30, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t29 to t30, the potential Pn is maintained at the potential at time t29, and the inversion signal S3 is maintained at low level. Furthermore, the periodic signal Sg2 is at low level during the period from time t29 to t30. Therefore, the detection signal S8 and the detection signal S9 are maintained at low level during the period from time t29 to t30.
[0059] At time t30, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logic threshold Th, and the inversion signal S3 transitions from low level to high level. Meanwhile, the periodic signal Sg1 and the periodic signal Sg2 are at low level at time t30. Thus, at time t30, the detection signal S8 and the detection signal S9 are at low level. Therefore, the incidence of the photon at time t30 does not cause the counting unit 551 and the counting unit 5522 to perform increment processing. That is, the count value held by the counting unit 551 is maintained at n+2, and the count value held by the counting unit 552 is maintained at m+1. Since the periodic signal S1 is maintained at high level during the period from time t30 to t31, the switch unit 520 is maintained in the off state. Therefore, the potential Pn is maintained at a potential less than the logic threshold Th at time t30, and the inversion signal S3 is maintained at a high level. Meanwhile, the periodic signals Sg1 and Sg2 are at a low level from time t30 to t31. Therefore, the detection signals S8 and S9 are maintained at a low level from time t30 to t31.
[0060] At time t31, period P4 ends and period P5 starts at the same time. The operation of each component and the change in each signal in the period from time t31 to t34 are the same as the operation of each component and the change in each signal in the period from time t1 to t4.
[0061] At time t34, the periodic signal Sg2 transitions from a low level to a high level, and is maintained at a high level during the period from time t34 to t35. Furthermore, since the periodic signal S1 is maintained at a high level during the period from time t34 to t35, the switch unit 520 is maintained in an off state. Therefore, the SPAD 510 is not charged during the period from time t34 to t35, the potential Pn is maintained at the potential at time t34, and the inverted signal S3 is maintained at a low level. Furthermore, the periodic signal Sg1 is at a low level during the period from time t34 to t35. Therefore, the detection signal S8 and the detection signal S9 are maintained at a low level during the period from time t34 to t35.
[0062] At time t35, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. The periodic signal Sg2 is at high level at time t35. Therefore, the detection signal S9 transitions from low level to high level at time t35. In response to the transition of the detection signal S9 to high level, an increment process is performed in the counter 552. That is, the counter 552 updates the count value from m+1 to m+2 at time t35. Meanwhile, since the periodic signal Sg1 is at low level at time t35, the detection signal S8 is at low level at time t35. Therefore, the increment process is not performed by the counter 551 at time t35, and the count value held by the counter 551 is maintained at n+2. Since the periodic signal S1 is maintained at a high level during the period from time t35 to t36, the switch section 520 is maintained in an off state. Therefore, during the period from time t35 to t36, the potential Pn is maintained at a potential less than the logical threshold value Th at time t35, and the inverted signal S3 is maintained at a high level. Furthermore, the periodic signal Sg2 is at a high level during the period from time t35 to t36. Therefore, during the period from time t35 to t36, the detection signal S9 is maintained at a high level. On the other hand, the periodic signal Sg1 is at a low level during the period from time t35 to t36. Therefore, during the period from time t35 to t36, the detection signal S8 is maintained at a low level.
[0063] The operation of each component and the change of each signal in the period from time t36 to t39 are the same as the operation of each component and the change of each signal in the period from time t12 to t15. Note that photons are incident on the SPAD 510 in the period from time t36 to t37. However, the SPAD 510 has already performed avalanche multiplication operation due to the incidence of photons at time t35, and the SPAD 510 is not charged in the period from time t35 to t37. Therefore, the potential Pn, the inversion signal S3, the detection signal S8, and the detection signal S9 are not affected by the incidence of photons in the period from time t36 to t37.
[0064] At time t39, the period P5 ends and at the same time, the period P6 starts. The operation of each component and the change of each signal at time t39 are the same as those at time t1. Since the periodic signal S1 is maintained at a low level during the period from time t39 to t40, the switch section 520 is maintained in an on state. Therefore, the SPAD 510 is charged during the period from time t39 to t40, but the potential Pn does not exceed the logic threshold value Th during the period from time t39 to t40. Therefore, the inversion signal S3 is maintained at a high level during the period from time t39 to t40. Meanwhile, since the periodic signals Sg1 and Sg2 are at a low level during the period from time t39 to t40, the detection signals S8 and S9 are maintained at a low level.
[0065] At time t40, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. The potential Pn drops in response to the avalanche multiplication operation of the SPAD 510. The SPAD 510 is charged again during the period from time t40 to t41, but the potential Pn is maintained below the logic threshold Th during the period from time t40 to t41. Therefore, the inversion signal S3 is maintained at a high level during the period from time t40 to t41. Meanwhile, the periodic signals Sg1 and Sg2 are at a low level during the period from time t40 to t41, and therefore the detection signals S8 and S9 are at a low level during the period from time t40 to t41. Therefore, the counting unit 551 and the increment process by the counting unit 551 are not executed during the period from time t40 to t41. That is, in the period from time t40 to t41, the count value held by counting unit 551 is maintained at n+3, and the count value held by counting unit 552 is maintained at m+2. Note that, in the period from time t40 to t41, photons are further incident on SPAD 510, and avalanche multiplication occurs in SPAD 510. Even in this case, the potential Pn is less than the logic threshold value Th in the period from time t39 to t41, as in the case where one photon is incident on SPAD 510 in the period from time t39 to t41. That is, counting units 551 and 552 operate independently of the number of times photons are incident in the period from time t39 to t41.
[0066] At time t41, the periodic signal S1 transitions from a low level to a high level, and the switch section 520 switches from an on state to an off state. Since the periodic signal S1 is maintained at a high level during the period from time t41 to t42, the switch section 520 is maintained in an off state. Therefore, the potential Pn is maintained at a potential less than the logical threshold value Th at time t41, and the inversion signal S3 is maintained at a high level. On the other hand, the periodic signals Sg1 and Sg2 are at a low level during the period from time t41 to t42. Therefore, the detection signals S8 and S9 are maintained at a low level during the period from time t41 to t42.
[0067] At time t42, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. The potential Pn drops in response to the avalanche multiplication operation of the SPAD 510, and is maintained at a potential less than the logical threshold Th. Therefore, the inversion signal S3 is maintained at a high level at time t42. Since the periodic signal S1 is maintained at a high level during the period from time t42 to t43, the switch unit 520 is maintained in an off state. Therefore, during the period from time t42 to t43, the potential Pn is maintained at a potential less than the logical threshold Th at time t42, and the inversion signal S3 is maintained at a high level. On the other hand, during the period from time t42 to t43, the periodic signals Sg1 and Sg2 are at a low level. Therefore, during the period from time t42 to t43, the detection signals S8 and S9 are maintained at a low level.
[0068] The operation of each component and the change of each signal during the period from time t43 to t47 are the same as those during the period from time t11 to t15. At time t47, period P6 ends. Note that photons are incident on the SPAD 510 during the period from time t43 to t44. However, the SPAD 510 has already performed avalanche multiplication due to the incidence of photons at time t42, and the SPAD 510 is not charged during the period from time t42 to t44. Therefore, the potential Pn, the inversion signal S3, the detection signal S8, and the detection signal S9 are not affected by the incidence of photons during the period from time t43 to t44.
[0069] As shown in FIG. 5, the circuit unit 530 generates the detection signal S8 based on the change in the voltage of the SPAD 510 and the periodic signal Sg1 from the time when the switch unit 520 transitions to the on state to the time when the periodic signal Sg1 transitions from the high level to the low level. When light is incident on the SPAD 510, the detection signal S8 transitions from the low level to the high level within the period when the periodic signal Sg1 is at the high level. The counter unit 551 increments the count value C1 at the time when the detection signal S8 transitions from the low level to the high level. On the other hand, the circuit unit 530 does not generate a high-level detection signal S8 based on the change in the voltage of the SPAD 510 after the time when the periodic signal Sg1 transitions from the high level to the low level during each of the periods P1 to P6. That is, only photons that are incident during the period from the time when the switch unit 520 transitions to the on state to the time when the periodic signal Sg1 transitions from the high level to the low level are subject to increment of the count value C1. Therefore, the counter 551 increments the count value C1 based on the photons incident on the SPAD 510 at time t10 of the period P2, at time t21 of the period P3, at time t35 of the period P5, and at time t40 of the period P6. On the other hand, in each of the periods P1 to P6, photons incident after the time when the periodic signal Sg1 transitions from high level to low level are not subject to increment of the count value C1. That is, the counter 551 does not increment the count value C1 based on the photons incident on the SPAD 510 at time t30 of the period P4. That is, the detection period Pd1 of the photons by the detection signal S8 from the circuit unit 530 is the period from the time when the periodic signal S1 transitions to low level to the time when the periodic signal Sg1 transitions from high level to low level (the shaded portion of Pd1 shown in FIG. 5).
[0070] Like the circuit unit 530, the circuit unit 540 generates the detection signal S9 based on the change in the voltage of the SPAD 510 and the periodic signal Sg2 from the time when the switch unit 520 transitions to the on state to the time when the periodic signal Sg2 transitions from the high level to the low level. That is, when light is incident on the SPAD 510, the detection signal S9 transitions from the low level to the high level within the period when the periodic signal Sg2 is at the high level. The counter unit 552 increments the count value C2 at the time when the detection signal S9 transitions from the low level to the high level. On the other hand, the circuit unit 540 does not generate a high-level detection signal S9 based on the change in the voltage of the SPAD 510 after the time when the periodic signal Sg2 transitions from the high level to the low level during each of the periods P1 to P6. That is, only photons that are incident during the period from the time when the switch unit 520 transitions to the on state to the time when the periodic signal Sg2 transitions from the high level to the low level are subject to increment of the count value C2. The periodic signal Sg2 has a predetermined phase difference with respect to the periodic signal Sg1. Therefore, in each of the periods P1 to P6, the time when the periodic signal Sg2 transitions from high level to low level is earlier than the time when the periodic signal Sg1 transitions from high level to low level. That is, the period during which the circuit unit 540 detects photons in each of the periods P1 to P6 is shorter than the period during which the circuit unit 530 detects photons. The counter 552 increments the count value C2 based on the photons that are incident on the SPAD 510 at time t10 in the period P2, time t35 in the period P5, and time t40 in the period P6. On the other hand, in each of the periods P1 to P6, photons that are incident after the time when the periodic signal Sg2 transitions from high level to low level are not subject to increment of the count value C2. That is, the counter 552 does not increment the count value C2 based on photons incident on the SPAD 510 at time t21 in period P3 and at time t30 in period P4. Therefore, the detection period Pd2 of photons by the detection signal S9 from the circuit unit 540 is the period from the time when the periodic signal S1 transitions to a low level to the time when the periodic signal Sg2 transitions from a high level to a low level (the shaded portion of Pd2 shown in FIG. 5).In addition, in periods P5 and P6, photons are incident on the SPAD 510 two or more times, but the counters 551 and 552 do not increment the count values C1 and C2 multiple times, but instead increment the count values C1 and C2 by only one, respectively.
[0071] The photoelectric conversion device 100 according to the present disclosure can provide different photon detection periods based on one signal. Specifically, based on one periodic signal S2 from the drive control unit 60, periodic signals Sg1 and Sg2 having different phases are input to the circuit unit 530 and the circuit unit 540, respectively. The photon detection period is determined by the phase difference between the periodic signal S2 and the periodic signals Sg1 and Sg2. That is, in each of the periods P1 to P6 corresponding to one period of the periodic signal S1, the time when the periodic signal Sg1 and the periodic signal Sg2 transition from a high level to a low level differs depending on the phase difference between the periodic signal Sg1 and the periodic signal Sg2. According to this embodiment, the time when the periodic signal Sg1 transitions from a high level to a low level is later than the time when the periodic signal Sg2 transitions from a high level to a low level. Therefore, the photon detection period counted by the counting unit 551 is longer than the photon detection period counted by the counting unit 552. That is, the photon detection sensitivity indicated by the detection signal S8 generated based on the periodic signal S2 is higher than the photon detection sensitivity indicated by the detection signal S9 generated based on the common periodic signal S2.
[0072] The photoelectric conversion device according to the present disclosure can obtain different photon detection sensitivities based on one periodic signal without requiring gating signals with different pulse widths for each measurement unit. That is, by using multiple signals whose phases are controlled based on one signal, photons can be counted at various sensitivities without complicating the circuit configuration of the photoelectric conversion device. Therefore, according to the present disclosure, the circuit configuration of the photoelectric conversion device capable of counting photons at multiple sensitivities can be further simplified.
[0073] The photoelectric conversion device according to the present disclosure can further generate a signal having a different phase from the periodic signals Sg1 and Sg2, and can measure the photons incident on the SPAD 510 based on the signal. That is, the photoelectric conversion device according to the present disclosure can count photons with various sensitivities without complicating the circuit configuration. Therefore, according to the present disclosure, the circuit configuration of the photoelectric conversion device can be further simplified and the dynamic range of the photoelectric conversion device can be expanded.
[0074] [Second embodiment] 6 is a circuit diagram of a pixel unit 50 according to the second embodiment. The pixel unit 50 according to this embodiment differs from the pixel unit 50 according to the first embodiment in that it further includes a determination unit 570 that determines whether or not the count values C1 and C2 exceed threshold values and specifies the count value to be selected by the selection unit 560 based on the result of the determination.
[0075] The determination unit 570 determines the count value output from the selection unit 560 based on the threshold signal Sth. As shown in FIG. 6, the determination unit 570 is connected to the selection unit 560 and receives the count value Cs output from the selection unit 560. The determination unit 570 also receives the threshold signal Sth output from the register block 40 via the setting signal line 12. The determination unit 570 compares the count value Cs with a threshold indicated by the threshold signal Sth. The determination unit 570 outputs a selection signal S4 to the selection unit 560 based on the result of the comparison between the count value Cs and the threshold. The selection unit 560 selects the value Cs based on the selection signal S4 and outputs it to the digital processing unit 70 via the output line 16. The determination unit 570 outputs the selection signal S4 to the digital processing unit 70 via the output line 16 to identify the selected count value and to identify the circuit unit that generated the selected count value.
[0076] For example, when the count value Cs is equal to or greater than the threshold value indicated by the threshold signal Sth, the determination unit 570 generates a selection signal S4 to designate the count value C2 from the counting unit 552, which has a relatively short photon detection period. The selection unit 560 selects the count value C2 output from the counting unit 552 as the count value Cs based on the selection signal S4 from the determination unit 570, and outputs the count value Cs to the digital processing unit 70 via the output line 16. On the other hand, when the count value Cs is less than the threshold value indicated by the threshold signal Sth, the determination unit 570 generates a selection signal S4 to designate the count value C1 from the counting unit 551, which has a relatively long photon detection period. The selection unit 560 selects the count value C1 output from the counting unit 551 as the count value Cs based on the selection signal S4 from the determination unit 570, and outputs the count value Cs to the digital processing unit 70 via the output line 16.
[0077] The number of bits of the threshold signal Sth may be the same as or different from the number of bits of the count value Cs. For example, the determination unit 570 may compare one bit at a predetermined position of the threshold signal Sth with one bit at a predetermined position of the count value Cs, and generate the selection signal S4 based on the result of the comparison. The number of bits of the selection signal S4 may be determined based on the number of counting units included in the pixel unit 50. For example, when the pixel unit 50 has two counting units, the selection signal S4 may be one bit. When the one-bit selection signal S4 indicates 0, the selection unit 560 may select the count value C1 output from the counting unit 551. When the one-bit selection signal S4 indicates 1, the selection unit 560 may select the count value C2 output from the counting unit 552. The determination unit 570 may set the initial value of the selection signal S4 to 0 (low level) or 1 (high level).
[0078] The operation of the determination unit 570 that determines the selection signal S4 based on the count value in one frame period and the threshold value of the count value will be described with reference to Figures 7A and 7B. Figures 7A and 7B are graphs showing the output of the count values output by the count units 551 and 552 in the second embodiment.
[0079] 7A shows a count value C551 counted by the counter 551 during a period from time t50 to t51 corresponding to one frame period, a count value C552 counted by the counter 552, a threshold value Cth of the count value indicated by the threshold signal Sth, and a selection signal S4 during the period from time t50 to t51. In FIG. 7A, the photoelectric conversion device 100 performs a photon detection operation in an environment with relatively low illuminance compared to FIG. 7B described later.
[0080] At time t50, the count values C551 and C552 are reset, and the selection signal S4 indicates 0 (low level). During the period from time t50 to t51, the count values C551 and C552 are both less than the threshold value Cth. Therefore, during the period from time t50 to t51, the selection signal S4 output from the determination unit 570 is maintained at 0 (low level) so as to designate the count value C551 from the counting unit 551 having a relatively long photon detection period.
[0081] 7B shows a count value C551 counted by the counter 551 during a period from time t52 to t55 corresponding to one frame period, a count value C552 counted by the counter 552, a count value threshold Cth indicated by the threshold signal Sth, and a selection signal S4 during the period from time t52 to t55. In FIG 7B, the photoelectric conversion device 100 performs a photon detection operation in an environment with a relatively high illuminance compared to FIG 7A.
[0082] At time t52, the count values C551 and C552 are reset, and the selection signal S4 indicates 0 (low level). During the period from time t52 to t53, both the count values C551 and C552 are less than the threshold value Cth. Therefore, during the period from time t52 to t53, the selection signal S4 output from the decision unit 570 indicates 0 (low level) to designate the count value C551 from the counting unit 551 having a relatively long photon detection period. At time t53, the count value C551 reaches the threshold value Cth. In response to the count value C551 reaching the threshold value Cth, the selection signal S4 switches from 0 (low level) to 1 (high level) at time 53 to designate the count value C552 from the counting unit 552 having a relatively short photon detection period. The selection signal S4 switched to 1 (high level) is output by the decision section 570 to the selection section 560 within one frame period (for example, at time t53).
[0083] During the period from time t53 to t54, the count values C551 and C552 increase, and at time t54, the count value C551 saturates. During the period from time t54 to t55, the count value C551 does not increase and is kept constant. Meanwhile, during the period from time t54 to t55, the count value C552 increases. At time t55, the count value C551 is equal to or greater than the threshold value Cth, and the count value C552 is less than the threshold value Cth. Therefore, during the period from time t53 to t55, the selection signal S4 output from the determination unit 570 is maintained at 1 (high level) so as to designate the count value C552 from the counting unit 552 having a relatively short photon detection period. Note that, if the count value C552 exceeds the threshold value Cth after time t53, the selection signal S4 is maintained at 1 (high level), and the count value C552 is output from the selection unit 560 as the count value Cs. Furthermore, the number of counting units included in the pixel unit 50 according to the present disclosure is not limited to 2. For example, in order to realize detection periods of three or more different photons, the pixel unit 50 can include three or more delay units and counting units, and the number of thresholds Cth can be set according to the number of counting units.
[0084] According to this embodiment, a count value (pixel value) output from the pixel unit 50 is selected based on count values output from a plurality of counters included in the pixel unit 50. That is, the photoelectric conversion device according to this embodiment does not require a selection signal to be generated outside the pixel unit 50, nor does it require the selection signal to be input to the pixel unit 50. Therefore, according to this embodiment, the circuit configuration of the photoelectric conversion device 100 can be simplified, and the power consumption of the photoelectric conversion device 100 can be reduced. Furthermore, the photoelectric conversion device 100 according to this embodiment can appropriately select an appropriate photon detection sensitivity depending on the amount of photons incident on the SPAD.
[0085] [Third embodiment] 8 is a circuit diagram of the pixel section 50 in the third embodiment. The pixel section 50 in this embodiment differs from the pixel section 50 in the first embodiment in that the pixel section 50 in this embodiment generates a detection signal S8 based on a periodic signal S1 input to a switch section 520. As shown in FIG. 8, the circuit section 530 in this embodiment outputs a detection signal S8 indicating detection of a photon in the SPAD 510 based on a logical AND operation of the inverted signal S3 at the node N2 and the periodic signal S1 (first periodic signal) from the drive control section 60. The circuit section 530 outputs a result of the operation based on the inverted signal S3 and the periodic signal S1 to the counter section 551 as the detection signal S8. Unlike the first and second embodiments, the pixel section 50 in this embodiment does not include a delay section 91.
[0086] FIG. 9 is a timing diagram showing the operation of the counting unit 551 and the counting unit 552 in the third embodiment. Each period of the periods P11 to P16 corresponds to one period of the periodic signal S1. FIG. 9 shows timing diagrams of the timing Tin of the incidence of photons to the SPAD 510 in the periods P11 to P16, the periodic signal S1 input to the gate of the switch unit 520, the potential Pn of the node N1, the inverted signal S3 input to the node N2, the detection period Pd1 of photons via the circuit unit 530, the periodic signal S1 input to the circuit unit 530, the detection signal S8 output from the circuit unit 530, the count value C1 output from the counting unit 551, the detection period Pd2 of photons via the circuit unit 540, the periodic signal Sg2 from the delay unit 92, the detection signal S9 output from the circuit unit 540, and the count value C2 output from the counting unit 552. The operation of the counting unit 552 in this embodiment is the same as the operation of the counting unit 552 shown in FIG. 5 in the first embodiment. Therefore, a description of the detection period Pd2 of photons via the circuit section 540, the periodic signal Sg2 from the delay section 92, the detection signal S9 output from the circuit section 540, and the count value C2 output from the counter section 552 will be omitted.
[0087] At time t60, which is before the period P11, the periodic signal S1 is at a high level. Also, at time t60, the potential Pn is less than the logic threshold Th, and the inversion signal S3 is at a high level. Therefore, at time t60, the detection signal S8 is at a high level. Also, at time t60, the count value C1 is n.
[0088] Since the periodic signal S1 is at a high level from time t60 to time t61, which is the start time of the period P11, the switch section 520 is maintained in an off state. Therefore, in the period from time t60 to t61, the SPAD 510 is not charged and the potential Pn is maintained below the logic threshold value Th. Therefore, in the period from time t60 to t61, the inverted signal S3 is maintained at a high level. Also, since the periodic signal S1 is at a high level in the period from time t60 to t61, the detection signal S8 is maintained at a high level.
[0089] At time t61, the periodic signal S1 transitions from a high level to a low level, and the switch unit 520 switches from an off state to an on state. When the switch unit 520 is turned on, charging of the SPAD 510 starts, and the detection period Pd1 in the period P11 starts. In addition, in response to the transition of the periodic signal S1 to a low level at time t61, the detection signal S8 transitions from a high level to a low level. Since the periodic signal S1 is maintained at a low level during the period from time t61 to t62, the switch unit 520 is maintained in an on state. Therefore, the SPAD 510 is charged during the period from time t61 to t62, but the potential Pn does not exceed the logic threshold Th during the period from time t61 to t62. Therefore, the inversion signal S3 is maintained at a high level during the period from time t61 to t62. Meanwhile, since the periodic signal S1 is at a low level during the period from time t61 to t62, the detection signal S8 is maintained at a low level.
[0090] At time t62, the potential Pn reaches the logic threshold Th, and the inversion signal S3 transitions from high to low. Since the periodic signal S1 is maintained at a low level during the period from time t62 to t63, the switch section 520 is maintained in an on state. That is, since charging of the SPAD 510 continues, the potential Pn rises beyond the logic threshold Th during the period from time t62 to t63. Therefore, the inversion signal S3 is maintained at a low level during the period from time t62 to t63. Therefore, the detection signal S8 is maintained at a low level during the period from time t62 to t63.
[0091] At time t63, the periodic signal S1 transitions from a low level to a high level, and the switch unit 520 switches from an on state to an off state. The switch unit 520 switches to an off state, and thus charging of the SPAD 510 stops. That is, the rise in the potential Pn stops. Since the periodic signal S1 is maintained at a high level during the period from time t63 to t64, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t63 to t64, the potential Pn is maintained at the potential at time t63, and the inversion signal S3 is maintained at a low level. Therefore, the detection signal S8 is maintained at a low level during the period from time t63 to t64.
[0092] At time t64, period P11 ends and period P12 starts at the same time. In the following description, the same operations of each component or changes in each signal as those already described may be omitted or simplified.
[0093] The operation of each component and the change of each signal at time t64 are the same as those at time t61. During the period from time t64 to t65, the periodic signal S1 is maintained at a low level, and therefore the switch section 520 is maintained in an on state. Although the SPAD 510 can be charged by the switch section 520 being in an on state, the SPAD 510 does not perform avalanche multiplication operation during the immediately preceding period P11. Therefore, the potential Pn is maintained at the potential at time t64 during the period from time t64 to t65. That is, the potential Pn is maintained at or above the logic threshold Th during the period from time t64 to t65, and the inversion signal S3 is maintained at a low level. Therefore, the detection signal S8 is maintained at a low level during the period from time t64 to t65.
[0094] At time t65, the periodic signal S1 transitions from low level to high level, and the switch section 520 switches from on to off. Since the periodic signal S1 is maintained at high level during the period from time t65 to t66, the switch section 520 is maintained in the off state. Therefore, the potential Pn is maintained at the potential at time t65, and the inversion signal S3 is maintained at low level. Therefore, the detection signal S8 is maintained at low level during the period from time t65 to t66.
[0095] At time t66, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. Also, at time t66, the periodic signal S1 is at high level. Therefore, the detection signal S8 transitions from low level to high level at time t66. In response to the transition of the detection signal S8 to high level, an increment process is performed in the counter 551. That is, the counter 551 updates the count value from n to n+1 at time t66. Since the periodic signal S1 is maintained at high level during the period from time t66 to t67, the switch unit 520 is maintained in the off state. Therefore, during the period from time t66 to t67, the potential Pn is maintained at a potential less than the logical threshold Th at time t66, and the inversion signal S3 is maintained at high level. Therefore, the detection signal S8 is maintained at a high level during the period from time t66 to time t67.
[0096] At time t67, period P12 ends and simultaneously period P13 starts. The operation of each component and the change in each signal in the period from time t67 to t69 are the same as the operation of each component and the change in each signal in the period from time t61 to t63.
[0097] At time t69, the periodic signal S1 transitions from a low level to a high level, and the switch unit 520 switches from an on state to an off state. The switch unit 520 switches to an off state, and thus charging of the SPAD 510 stops. That is, the rise in the potential Pn stops. Since the periodic signal S1 is maintained at a high level during the period from time t69 to t70, the switch unit 520 is maintained in the off state. Therefore, the SPAD 510 is not charged during the period from time t69 to t70, the potential Pn is maintained at the potential at time t69, and the inversion signal S3 is maintained at a low level. Therefore, the detection signal S8 is maintained at a low level during the period from time t69 to t70.
[0098] At time t70, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. The periodic signal S1 is at high level at time t70. Therefore, at time t70, the detection signal S8 transitions from low level to high level. In response to the transition of the detection signal S8 to high level, an increment process is performed in the counter 551. That is, the counter 551 updates the count value from n+1 to n+2 at time t70. Since the periodic signal S1 is maintained at high level during the period from time t70 to t71, the switch unit 520 is maintained in the off state. Therefore, in the period from time t70 to t71, the potential Pn is maintained at a potential less than the logic threshold Th at time t21, and the inversion signal S3 is maintained at a high level, and therefore, in the period from time t70 to t71, the detection signal S8 is maintained at a high level.
[0099] At time t71, period P13 ends and period P14 starts at the same time. The operation of each component and the change in each signal in the period from time t71 to t74 are the same as the operation of each component and the change in each signal in the period from time t67 to t70.
[0100] At time t74, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. The periodic signal S1 is at high level at time t74. Therefore, at time t74, the detection signal S8 transitions from low level to high level. In response to the transition of the detection signal S8 to high level, an increment process is performed in the counter 551. That is, the counter 551 updates the count value from n+2 to n+3 at time t74. The operation of each component and the change of each signal during the period from time t74 to t75 are the same as the operation of each component and the change of each signal during the period from time t70 to t71.
[0101] At time t75, period P14 ends and period P15 starts at the same time. The operation of each component and the change in each signal in the period from time t75 to t78 are the same as the operation of each component and the change in each signal in the period from time t67 to t70.
[0102] At time t78, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. In response to the avalanche multiplication operation of the SPAD 510, the potential Pn becomes less than the logical threshold Th, and the inversion signal S3 transitions from low level to high level. The periodic signal S1 is at high level at time t78. Therefore, at time t78, the detection signal S8 transitions from low level to high level. In response to the transition of the detection signal S8 to high level, an increment process is performed in the counter 551. That is, the counter 551 updates the count value from n+3 to n+4 at time t78. Since the periodic signal S1 is maintained at high level during the period from time t78 to t79, the switch unit 520 is maintained in the off state. Therefore, in the period from time t78 to t79, the potential Pn is maintained at a potential less than the logic threshold Th at time t78, and the inversion signal S3 is maintained at a high level. Therefore, in the period from time t78 to t79, the detection signal S8 is maintained at a high level. Note that photons are incident on the SPAD 510 in the period from time t78 to t79. However, the SPAD 510 has already performed avalanche multiplication operation due to the incidence of photons at time t78, and the SPAD 510 is not charged in the period from time t78 to t79. Therefore, the potential Pn, the inversion signal S3, and the detection signal S8 are not affected by the incidence of photons in the period from time t78 to t79.
[0103] At time t79, the period P15 ends and at the same time, the period P16 starts. The operation of each component and the change of each signal at time t79 are the same as those at time t61. Since the periodic signal S1 is maintained at a low level during the period from time t79 to t80, the switch section 520 is maintained in an on state. Therefore, the SPAD 510 is charged during the period from time t79 to t80, but the potential Pn does not exceed the logic threshold value Th during the period from time t79 to t80. Therefore, the inversion signal S3 is maintained at a high level during the period from time t79 to t80. Meanwhile, since the periodic signal S1 is at a low level during the period from time t79 to t80, the detection signal S8 is maintained at a low level.
[0104] At time t80, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. The potential Pn drops in response to the avalanche multiplication operation of the SPAD 510. The SPAD 510 is charged again during the period from time t80 to t81, but the potential Pn is maintained below the logic threshold Th during the period from time t80 to t81. Therefore, the inversion signal S3 is maintained at a high level during the period from time t80 to t81. On the other hand, since the periodic signal S1 is at a low level during the period from time t80 to t81, the detection signal S8 is at a low level during the period from time t80 to t81. Therefore, the counting unit 551 does not perform an increment process during the period from time t80 to t81. That is, during the period from time t80 to t81, the count value held by the counting unit 551 is maintained at n+4. In the period from time t80 to t81, photons are further incident on the SPAD 510, and avalanche multiplication occurs in the SPAD 510. Even in this case, the potential Pn is less than the logic threshold Th in the period from time t79 to t81, as in the case where one photon is incident on the SPAD 510 in the period from time t79 to t81. Therefore, the counting unit 551 operates independently of the number of times photons are incident in the period from time t79 to t81.
[0105] At time t81, the periodic signal S1 transitions from a low level to a high level, and the switch unit 520 switches from an on state to an off state. Also, the inverted signal S3 is at a high level at time t81. Therefore, the detection signal S8 transitions from a low level to a high level at time t81. In response to the detection signal S8 transitioning to a high level, the counter unit 551 executes an increment process. That is, the counter unit 551 updates the count value from n+4 to n+5 at time t81. Since the periodic signal S1 is maintained at a high level during the period from time t81 to t82, the switch unit 520 is maintained in an off state. Therefore, the potential Pn is maintained at a potential less than the logic threshold value Th at time t81, and the inverted signal S3 is maintained at a high level. Therefore, the detection signal S8 is maintained at a high level during the period from time t81 to t82.
[0106] At time t82, a photon is incident on the SPAD 510. The incidence of the photon causes avalanche multiplication in the SPAD 510. The potential Pn drops in response to the avalanche multiplication operation of the SPAD 510, and is maintained at a potential lower than the logical threshold Th. Therefore, the inversion signal S3 and the detection signal S8 are maintained at a high level at time t82. The operation of each component and the change of each signal during the period from time t82 to t83 are the same as the operation of each component and the change of each signal during the period from time t78 to t79. Note that, during the period from time t82 to t83, a photon is incident on the SPAD 510. However, the SPAD 510 has already performed avalanche multiplication operation due to the incidence of the photon at time t82, and the SPAD 510 is not charged during the period from time t82 to t83. Therefore, the potential Pn, the inversion signal S3, and the detection signal S8 are not affected by the incidence of the photon during the period from time t82 to t83. At time t83, the period P16 ends.
[0107] As shown in FIG. 9, the circuit unit 530 in this embodiment generates the detection signal S8 based on the voltage change of the SPAD 510 over one period of the periodic signal S1 and the periodic signal S1 input to the circuit unit 530. That is, the circuit unit 530 generates the detection signal S8 based on the voltage change of the SPAD 510 over the period in which the switch unit 520 is in the on state and the off state. When light is incident on the SPAD 510, the detection signal S8 transitions from a low level to a high level within the period in which the periodic signal S1 is at a high level. The counter unit 551 increments the count value C1 at the time when the detection signal S8 transitions from a low level to a high level. Thus, the counter unit 551 increments the count value C1 based on the photons incident on the SPAD 510 at time t66 in the period P12, time t70 in the period P13, time t74 in the period P14, time t78 in the period P15, and time t80 in the period P16. Therefore, the detection period Pd1 of photons by the detection signal S8 from the circuit section 530 is the entire period of one period of the periodic signal S1 (the shaded portion of Pd1 shown in FIG. 9).
[0108] The pixel unit 50 in this embodiment outputs the count value C1 without receiving the input of the periodic signal Sg1. That is, the photoelectric conversion device 100 in this embodiment can output two count values C1 and C2 based on different photon sensitivities without using the delay unit 91 in the first embodiment. Therefore, according to this embodiment, it is not necessary to provide the delay unit 91 and wiring for supplying the periodic signal S2 to the delay unit 91 from outside the pixel unit 50 to each of the pixel units 50. Therefore, according to this embodiment, the circuit configuration of the photoelectric conversion device can be further simplified, the photoelectric conversion device can be further miniaturized, and further reduction in power consumption can be achieved.
[0109] [Fourth embodiment] The configuration of the photoelectric conversion device in this embodiment will be described with reference to Figs. 10 to 13. The photoelectric conversion device has a SPAD pixel including an avalanche photodiode (hereinafter referred to as "APD"). The conductivity type of the charge used as the signal charge among the charge pairs generated in the APD is called the first conductivity type. The first conductivity type refers to a conductivity type in which charges of the same polarity as the signal charge are the majority carriers. The conductivity type opposite to the first conductivity type is called the second conductivity type. In the following, an example will be described in which the signal charge is an electron, the first conductivity type is an N type, and the second conductivity type is a P type, but the signal charge may be a hole, the first conductivity type is a P type, and the second conductivity type is an N type.
[0110] FIG. 10 is a schematic diagram of a photoelectric conversion device in this embodiment, showing the configuration of a stacked photoelectric conversion device 100. The photoelectric conversion device 100 includes a sensor substrate (first substrate) 1 and a circuit substrate (second substrate) 2 stacked on each other, and the sensor substrate 1 and the circuit substrate 2 are electrically connected to each other. The photoelectric conversion device in this embodiment is a back-illuminated photoelectric conversion device in which light is incident from a first surface of the sensor substrate 1 and the circuit substrate 2 is disposed on a second surface of the sensor substrate 1. The sensor substrate 1 has a first semiconductor layer having a photoelectric conversion element described later and a first wiring structure. The circuit substrate 2 has a second semiconductor layer having a circuit such as a signal processing unit described later and a second wiring structure. The second semiconductor layer, the second wiring structure, the first wiring structure, and the first semiconductor layer are stacked in this order to form the photoelectric conversion device 100.
[0111] In the following, the sensor substrate 1 and the circuit substrate 2 may be diced chips, but are not limited to chips. For example, each substrate may be a wafer. Also, each substrate may be stacked in a wafer state and then diced, or may be chipped and then stacked and bonded. The sensor substrate 1 is provided with a pixel region 1a, and the circuit substrate 2 is provided with a circuit region 2a that processes signals detected by the pixel region 1a.
[0112] 11 is a diagram showing an example of the arrangement of the sensor substrate 1. A plurality of pixels 10 each include an APD 11, and are arranged in a two-dimensional array in a plan view to form a pixel region 1a.
[0113] The pixel 10 is typically a pixel for forming an image, but when used for TOF (Time of Flight), it does not necessarily have to form an image. That is, the pixel 10 may be a pixel for measuring the time and amount of light that arrives.
[0114] FIG. 12 is a diagram showing an example of the arrangement of the circuit board 2. On the circuit board 2, a signal processing unit 20, a vertical scanning circuit 21, a readout circuit 23, a horizontal scanning circuit 27, an output calculation unit 24, a control pulse generating circuit 25, scanning lines 26, and signal lines 29 are formed. A circuit area 2a is arranged in an area overlapping the pixel area 1a of FIG. 11 in a plan view. Furthermore, the vertical scanning circuit 21, the readout circuit 23, the horizontal scanning circuit 27, the output calculation unit 24, and the control pulse generating circuit 25 are arranged so as to overlap an area between the end of the sensor substrate 1 and the end of the pixel area 1a of FIG. 11 in a plan view. That is, the sensor substrate 1 has the pixel area 1a and a non-pixel area arranged around the pixel area 1a, and the vertical scanning circuit 21, the readout circuit 23, the horizontal scanning circuit 27, the output calculation unit 24, and the control pulse generating circuit 25 are arranged in an area overlapping the non-pixel area in a plan view.
[0115] The signal processing unit 20 is electrically connected to the pixels 10 via connection wiring provided for each pixel 10, and is arranged in a two-dimensional array in a planar view, similar to the pixels 10. The signal processing unit 20 includes a binary counter that counts photons incident on the pixels 10.
[0116] The vertical scanning circuit 21 receives a control pulse supplied from a control pulse generating circuit 25, and supplies the control pulse to the signal processing unit 20 corresponding to the pixels 10 in each row via a scanning line 26. The vertical scanning circuit 21 can be composed of logic circuits such as a shift register and an address decoder.
[0117] The readout circuit 23 obtains the pulse count value of the digital signal from the signal processing unit 20 of each row via the signal line 29. Then, the readout circuit 23 outputs the output signal to a signal processing circuit (signal processing device) outside the photoelectric conversion device 100 via the output calculation unit 24. The readout circuit 23 may also have a function of a signal processing circuit that corrects the count value. The horizontal scanning circuit 27 receives a control pulse from the control pulse generation circuit 25, and causes the output calculation unit 24 to sequentially output the count value of each column in the readout circuit 23. As described later, when the pulse count value exceeds a threshold value, the output calculation unit 24 estimates an actual image signal (pulse count value) based on the time count value and threshold value included in the additional information, and replaces (extrapolates) the pulse count value with the estimated pulse count value. On the other hand, when the pulse count value is equal to or less than the threshold value, the pulse count value is output as it is as an image signal.
[0118] The output calculation unit 24 performs a predetermined process on the pulse count value read by the readout circuit 23 and outputs an image signal to the outside. As will be described later, the output calculation unit 24 can execute processes such as calculation of the pulse count value when the pulse count value exceeds a threshold value.
[0119] In FIG. 11, the photoelectric conversion elements in the pixel region 1a may be arranged one-dimensionally. The effect of the present disclosure can be achieved even in a configuration with one pixel 10, and the configuration with one pixel 10 may also be included in the present disclosure. In a photoelectric conversion device having a plurality of pixels 10, the effect of suppressing the circuit size according to this embodiment becomes even more remarkable. It is not necessary to provide one signal processing unit 20 for each pixel 10. For example, one signal processing unit 20 may be shared by a plurality of pixels 10, and signal processing may be performed sequentially.
[0120] Fig. 13 is a block diagram of the APD and pulse generating unit in this embodiment. Fig. 13 shows a pixel 10 of a sensor substrate 1 and a pulse generating unit 22 in a signal processing unit 20 of a circuit substrate 2. An APD 11 is disposed in the pixel 10. The pulse generating unit 22 includes a quenching element 221, a waveform shaping unit 222, a counter circuit 223, and a selection circuit 224.
[0121] The APD 11 generates pairs of charges according to the incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD 11. A voltage VH (second voltage) higher than the voltage VL supplied to the anode is supplied to the cathode of the APD 11. A reverse bias voltage is applied to the anode and cathode, and the APD 11 is in a state where avalanche multiplication is possible. When a photon is incident on the APD 11 while the reverse bias voltage is being supplied, the charge generated by the photon undergoes avalanche multiplication, generating an avalanche current.
[0122] Depending on the reverse bias voltage, the APD 11 can operate in Geiger mode or linear mode. The Geiger mode is an operation in a state where the potential difference between the anode and the cathode is greater than the breakdown voltage, and the linear mode is an operation in a state where the potential difference between the anode and the cathode is close to or less than the breakdown voltage. An APD operating in Geiger mode is particularly called a SPAD or SPAD type. As an example, the voltage VL (first voltage) can be −30V, and the voltage VH (second voltage) can be 1V. The APD 11 can operate in either linear mode or Geiger mode. When the APD 11 operates as a SPAD, the potential difference becomes larger compared to the APD 11 in linear mode, and the effect of withstanding voltage becomes more pronounced, so it is preferable that the APD 11 is a SPAD.
[0123] The quench element 221 is provided between a power supply line that supplies the voltage VH and the cathode of the APD 11. The quench element 221 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication, and has a function of suppressing avalanche multiplication by suppressing the voltage supplied to the APD 11 (quench operation). The quench element 221 also has a function of returning the voltage supplied to the APD 11 to the voltage VH by flowing a current equivalent to the voltage drop caused by the quench operation (recharge operation).
[0124] The waveform shaping unit 222 functions as a signal generating unit that generates a detection pulse based on an output generated by the incidence of a photon. That is, the waveform shaping unit 222 shapes the potential change of the cathode of the APD 11 obtained at the time of photon detection, and outputs a square wave pulse signal (detection pulse). For example, an inverter circuit is used as the waveform shaping unit 222. Although an example using one inverter as the waveform shaping unit 222 is shown in FIG. 13, a circuit in which a plurality of inverters are connected in series may be used. Also, other circuits having a waveform shaping effect may be used.
[0125] The counter circuit 223 counts the pulse signal output from the waveform shaping unit 222 and holds the count value. In addition, a control pulse is supplied to the counter circuit 223 from the vertical scanning circuit 21 in Fig. 12 via a driving line 226 of the scanning line 26. When the control pulse becomes active, the signal held in the counter circuit 223 is reset.
[0126] The selection circuit 224 includes a switch circuit, a buffer circuit for outputting a signal, etc. A control pulse is supplied to the selection circuit 224 from the vertical scanning circuit 21 in Fig. 12 via a drive line 227. In response to the control pulse, the selection circuit 224 switches between electrical connection and non-connection between the counter circuit 223 and the signal line 219.
[0127] Switches such as transistors may be provided between the quench element 221 and the APD 11, and between the APD 11 and the signal processing unit 20. The supply of the voltage VH or the voltage VL may be electrically switched by a switch such as a transistor.
[0128] Fig. 14 is a diagram showing the relationship between the operation of the APD and the output signal in this embodiment. Fig. 14(a) is a diagram showing the APD 11, quench element 221, and waveform shaping unit 222 of Fig. 13. When the input side of the waveform shaping unit 222 is nodeA and the output side is nodeB, Fig. 14(b) shows the waveform change of nodeA, and Fig. 14(c) shows the waveform change of nodeB.
[0129] Between time t0 and time t1, a reverse bias voltage of VH-VL is applied to the APD 11. When a photon is incident on the APD 11 at time t1, avalanche multiplication occurs in the APD 11, an avalanche multiplication current flows through the quench element 221, and the voltage of nodeA drops. When the voltage drop becomes larger and the potential difference applied to the APD 11 becomes smaller, the avalanche multiplication of the APD 11 stops at time t3, and the voltage level of nodeA does not drop by more than a certain value. After that, between time t3 and time t5, a current that compensates for the voltage drop flows from the voltage VL to nodeA, and at time t5, nodeA is stabilized to the original voltage level. At this time, when the voltage level of nodeA falls below the threshold of the waveform shaping unit 222 from time t2 to time t4, nodeB becomes high level. That is, the voltage waveform of nodeA is shaped by the waveform shaping section 222, and a rectangular wave pulse signal is output from nodeB.
[0130] [Fifth embodiment] FIG. 15 is a diagram of a light detection system in this embodiment, and is a block diagram of a range image sensor using the photoelectric conversion device described in the above embodiment.
[0131] 15, the range image sensor 401 includes an optical system 402, a photoelectric conversion device 403, an image processing circuit 404, a monitor 405, and a memory 406. The range image sensor 401 receives light (modulated light, pulsed light) emitted from a light source device 411 toward a subject and reflected from the surface of the subject. The range image sensor 401 can obtain a range image according to the distance to the subject, based on the time from emission to reception of light.
[0132] The optical system 402 includes one or more lenses, guides image light (incident light) from a subject to the photoelectric conversion device 403 , and forms an image on the light receiving surface (sensor portion) of the photoelectric conversion device 403 .
[0133] The photoelectric conversion device according to each of the above-mentioned embodiments can be applied as the photoelectric conversion device 403. The photoelectric conversion device 403 supplies the image processing circuit 404 with a distance signal indicating a distance determined from a received light signal.
[0134] The image processing circuit 404 performs image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 403. The distance image (image data) obtained by the image processing can be displayed on a monitor 405 and stored (recorded) in a memory 406.
[0135] The range image sensor 401 configured in this way can obtain a more accurate range image as the pixel characteristics are improved by applying the above-mentioned photoelectric conversion device.
[0136] [Sixth embodiment] The technology according to the present disclosure may be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.
[0137] Fig. 16 is a schematic diagram of the endoscopic surgery system in this embodiment. Fig. 16 shows a state in which an operator (doctor) 1131 performs surgery on a patient 1132 on a patient bed 1133 using an endoscopic surgery system 1103. As shown in the figure, the endoscopic surgery system 1103 includes an endoscope 1100, a surgical tool 1110, and a cart 1134 on which various devices for endoscopic surgery are mounted.
[0138] The endoscope 1100 includes a lens barrel 1101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 1132, a camera head 1102 connected to a base end of the lens barrel 1101, and an arm 1121. Although Fig. 16 shows the endoscope 1100 configured as a so-called rigid scope having a rigid lens barrel 1101, the endoscope 1100 may also be configured as a so-called flexible scope having a flexible lens barrel.
[0139] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 1101. A light source device 1203 is connected to the endoscope 1100, and light generated by the light source device 1203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 1101, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 1132. The endoscope 1100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
[0140] An optical system and a photoelectric conversion device are provided inside the camera head 1102, and reflected light (observation light) from an observation target is focused on the photoelectric conversion device by the optical system. The observation light is photoelectrically converted by the photoelectric conversion device to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. As the photoelectric conversion device, the photoelectric conversion device described in each of the above-mentioned embodiments may be used. The image signal is transmitted to a camera control unit (CCU) 1135 as RAW data.
[0141] The CCU 1135 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 1100 and the display device 1136. Furthermore, the CCU 1135 receives an image signal from the camera head 1102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.
[0142] Under the control of the CCU 1135 , the display device 1136 displays an image based on an image signal that has been subjected to image processing by the CCU 1135 .
[0143] The light source device 1203 includes a light source such as an LED (Light Emitting Diode), and supplies the endoscope 1100 with irradiation light when imaging an operation site or the like.
[0144] The input device 1137 is an input interface for the endoscopic surgery system 1103. A user can input various information and instructions to the endoscopic surgery system 1103 via the input device 1137.
[0145] The treatment tool control device 1138 controls the driving of the energy treatment tool 1112 for cauterizing tissue, incising, sealing blood vessels, or the like.
[0146] The light source device 1203 can supply irradiation light to the endoscope 1100 when imaging the surgical site, and can be, for example, a white light source made of an LED, a laser light source, or a combination of these. When a white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. Therefore, the light source device 1203 can adjust the white balance of the captured image. In this case, the laser light from each of the RGB laser light sources can be irradiated to the observation target in a time-division manner, and the drive of the image sensor of the camera head 1102 can be controlled in synchronization with the irradiation timing. This makes it possible to capture images corresponding to each of the RGB colors in a time-division manner. According to this method, a color image can be obtained without providing a color filter to the image sensor.
[0147] Furthermore, the driving of the light source device 1203 may be controlled so that the intensity of the light output from the light source device 1203 is changed at predetermined time intervals. By controlling the driving of the image sensor of the camera head 1102 in synchronization with the timing of the change in the light intensity to acquire images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blocked-up shadows and blown-out highlights.
[0148] Furthermore, the light source device 1203 may be configured to be capable of supplying light of a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue can be utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface can be photographed with high contrast by irradiating light of a narrower band than the irradiation light (i.e., white light) during normal observation. Alternatively, in the special light observation, a fluorescent observation may be performed in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 1203 may be configured to be capable of supplying narrow band light and / or excitation light corresponding to such special light observation.
[0149] [Seventh embodiment] The light detection system and the moving body of this embodiment will be described with reference to Figures 17A, 17B, and 18. In this embodiment, an example of an on-vehicle camera will be shown as the light detection system.
[0150] FIG. 17A is a schematic diagram of a light detection system in this embodiment, showing an example of a vehicle system and a light detection system mounted on the vehicle system. The light detection system 1301 includes a photoelectric conversion device 1302, an image preprocessing unit 1315, an integrated circuit 1303, and an optical system 1314. The optical system 1314 forms an optical image of a subject on the photoelectric conversion device 1302. The photoelectric conversion device 1302 converts the optical image of the subject formed by the optical system 1314 into an electrical signal. The photoelectric conversion device 1302 is any of the photoelectric conversion devices in each of the above-mentioned embodiments. The image preprocessing unit 1315 performs a predetermined signal processing on the signal output from the photoelectric conversion device 1302. The function of the image preprocessing unit 1315 may be incorporated in the photoelectric conversion device 1302. The light detection system 1301 is provided with at least two sets of an optical system 1314 , a photoelectric conversion device 1302 , and an image pre-processing unit 1315 , and the output from each set of the image pre-processing unit 1315 is input to the integrated circuit 1303 .
[0151] The integrated circuit 1303 is an integrated circuit for use in an imaging system, and includes an image processing unit 1304 including a storage medium 1305, an optical distance measuring unit 1306, a parallax calculation unit 1307, an object recognition unit 1308, and an abnormality detection unit 1309. The image processing unit 1304 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 1315. The storage medium 1305 stores the primary storage of the captured image and the defective positions of the captured pixels. The optical distance measuring unit 1306 performs focusing and distance measurement of the subject. The parallax calculation unit 1307 calculates distance measurement information from multiple image data acquired by multiple photoelectric conversion devices 1302. The object recognition unit 1308 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 1309 detects an abnormality in the photoelectric conversion device 1302, it notifies the main control unit 1313 of the abnormality.
[0152] The integrated circuit 1303 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.
[0153] The main control unit 1313 (mobile object control unit) supervises and controls the operations of the light detection system 1301, the vehicle sensor 1310, the control unit 1320, etc. The main control unit 1313 may not be provided, and the light detection system 1301, the vehicle sensor 1310, and the control unit 1320 may each have a communication interface and each may transmit and receive control signals via a communication network according to, for example, the CAN standard.
[0154] The integrated circuit 1303 has a function of receiving a control signal from the main control unit 1313 or transmitting a control signal or a set value to the photoelectric conversion device 1302 by its own control unit.
[0155] The optical detection system 1301 is connected to a vehicle sensor 1310, and can detect the vehicle's driving state, such as vehicle speed, yaw rate, and steering angle, as well as the state of the environment outside the vehicle, other vehicles, and obstacles. The vehicle sensor 1310 is also a distance information acquisition unit that acquires distance information to an object. The optical detection system 1301 is also connected to a driving assistance control unit 1311 that performs various driving assistance functions, such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 1301 and the vehicle sensor 1310. This allows for avoidance control when a collision is estimated, and activation of a safety device when a collision occurs.
[0156] The light detection system 1301 is also connected to an alarm device 1312 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 1313 performs vehicle control such as applying the brakes, releasing the accelerator, and suppressing engine output to avoid a collision and reduce damage. The alarm device 1312 issues an alarm to the user using means such as issuing an alarm such as a sound, displaying alarm information on the display screen of a car navigation system or a meter panel, or applying vibration to a seat belt or steering wheel.
[0157] The light detection system 1301 in this embodiment can capture an image of the surroundings of the vehicle, for example, the front or rear of the vehicle. Fig. 17B is a schematic diagram of a moving body in this embodiment, and shows a configuration in which the light detection system 1301 captures an image of the area in front of the vehicle.
[0158] The two photoelectric conversion devices 1302 are disposed in front of the vehicle 1300. Specifically, it is preferable to regard the center line of the vehicle 1300 relative to the forward / backward direction or the outer shape (for example, the vehicle width) as an axis of symmetry, and to dispose the two photoelectric conversion devices 1302 in line symmetry with respect to the axis of symmetry. This makes it possible to effectively obtain distance information between the vehicle 1300 and an object to be photographed and to determine the possibility of a collision. In addition, it is preferable that the photoelectric conversion device 1302 is disposed in a position that does not obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 1300 from the driver's seat. It is preferable that the alarm device 1312 is disposed in a position that is easily within the driver's field of vision.
[0159] Next, a fault detection operation of the photoelectric conversion device 1302 in the photodetection system 1301 will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the operation of the photodetection system in this embodiment. The fault detection operation of the photoelectric conversion device 1302 can be executed in accordance with steps S1410 to S1480.
[0160] In step S1410, startup settings are made for the photoelectric conversion device 1302. That is, setting information for the operation of the photoelectric conversion device 1302 is transmitted from outside the photodetection system 1301 (e.g., the main control unit 1313) or from inside the photodetection system 1301, and the photoelectric conversion device 1302 starts imaging operation and fault detection operation.
[0161] Next, in step S1420, the photoelectric conversion device 1302 acquires pixel signals from the valid pixels. In addition, in step S1430, the photoelectric conversion device 1302 acquires output values from failure detection pixels provided for failure detection. The failure detection pixels include photoelectric conversion elements, just like the valid pixels. A predetermined voltage is written to the photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to the photoelectric conversion elements. Note that steps S1420 and S1430 may be executed in the reverse order.
[0162] Next, in step S1440, the photodetection system 1301 performs a correspondence determination between the output expected value of the fault detection pixel and the actual output value from the fault detection pixel. If the correspondence determination result in step S1440 indicates that the output expected value and the actual output value match, the photodetection system 1301 proceeds to processing in step S1450, determines that the imaging operation is performed normally, and proceeds to processing in step S1460. In step S1460, the photodetection system 1301 transmits the pixel signal of the scanning row to the storage medium 1305 and temporarily stores it. Thereafter, the photodetection system 1301 returns to the processing in step S1420 and continues the fault detection operation. On the other hand, if the correspondence determination result in step S1440 indicates that the output expected value and the actual output value do not match, the photodetection system 1301 proceeds to processing in step S1470. In step S1470, the light detection system 1301 determines that there is an abnormality in the imaging operation, and issues an alarm to the main control unit 1313 or the alarm device 1312. The alarm device 1312 displays on the display unit that an abnormality has been detected. After that, in step S1480, the light detection system 1301 stops the photoelectric conversion device 1302, and ends the operation of the light detection system 1301.
[0163] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S1470 may be notified to the outside of the vehicle via a wireless network.
[0164] In addition, in the present embodiment, the control to prevent collision with other vehicles has been described, but the present disclosure can also be applied to control to automatically drive by following other vehicles, control to automatically drive without going out of lanes, and the like. Furthermore, the light detection system 1301 can be applied not only to vehicles such as the vehicle itself, but also to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the light detection system 1301 can be applied not only to moving bodies, but also to devices that widely use object recognition, such as intelligent transport systems (ITS). The photoelectric conversion device of the present disclosure may further be configured to be capable of acquiring various information such as distance information.
[0165] [Eighth embodiment] FIG. 19(a) is a diagram showing a specific example of an electronic device in this embodiment, showing glasses 1600 (smart glasses). The glasses 1600 are provided with a photoelectric conversion device 1602 described in each of the above-mentioned embodiments. A display device including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 1601. The photoelectric conversion device 1602 may be one or more. Also, multiple types of photoelectric conversion devices may be combined. The arrangement position of the photoelectric conversion device 1602 is not limited to that shown in FIG. 19(a).
[0166] The glasses 1600 further include a control device 1603. The control device 1603 functions as a power source that supplies power to the photoelectric conversion device 1602 and the above-mentioned display device. The control device 1603 also controls the operations of the photoelectric conversion device 1602 and the display device. The lens 1601 is formed with an optical system for collecting light on the photoelectric conversion device 1602.
[0167] FIG. 19(b) shows glasses 1610 (smart glasses) according to one application example. The glasses 1610 have a control device 1612, and the control device 1612 is equipped with a photoelectric conversion device corresponding to the photoelectric conversion device 1602 and a display device. The lens 1611 is formed with a photoelectric conversion device in the control device 1612 and an optical system for projecting light emitted from the display device, and an image is projected onto the lens 1611. The control device 1612 functions as a power source that supplies power to the photoelectric conversion device and the display device, and controls the operation of the photoelectric conversion device and the display device. The control device 1612 may have a line of sight detection unit that detects the line of sight of the wearer. Infrared light may be used to detect the line of sight. The infrared light emission unit emits infrared light toward the eyeball of a user gazing at a display image. An image of the eyeball is obtained by detecting the reflected light of the emitted infrared light from the eyeball with an imaging unit having a light receiving element. By providing a reduction unit that reduces the amount of light from the infrared light emitting unit to the display unit in a plan view, degradation of image quality is reduced.
[0168] The gaze of the user with respect to the displayed image is detected from an image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using the image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used.
[0169] More specifically, a gaze detection process based on the pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector that indicates the direction (rotation angle) of the eyeball is calculated based on the pupil image and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.
[0170] The display device of this embodiment may have a photoelectric conversion device having a light receiving element, and may control the display image of the display device based on information on the user's line of sight from the photoelectric conversion device.
[0171] Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. In the display area of the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than that of the first field of view area.
[0172] The display area may include a first display area and a second display area different from the first display area. A high priority area may be determined from the first display area and the second display area based on line-of-sight information. The first field of view area and the second field of view area may be determined by a control device of the display device or an external control device. The resolution of the high priority area may be controlled to be higher than the resolution of areas other than the high priority area. In other words, the resolution of an area with a relatively low priority may be lowered.
[0173] In addition, AI (Artificial Intelligence) may be used in determining the first field of view area and the area with high priority. The AI may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using the image of the eyeball and the direction in which the eyeball in the image was actually looking as teacher data. The AI program may be provided in either the display device or the photoelectric conversion device, or may be provided in an external device. When the external device has the AI program, it may be transmitted from a server or the like to the display device via communication.
[0174] In the case where display control is performed based on visual recognition detection, the present embodiment can be preferably applied to smart glasses further including a photoelectric conversion device that captures an image of the outside world. The smart glasses can display captured outside information in real time.
[0175] [Other embodiments] The present disclosure is not limited to the above-described embodiments and may be modified in various ways. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present disclosure.
[0176] The disclosure of this specification includes the following configurations. (Configuration 1) A detection unit that generates a voltage change in response to the incidence of a photon; a switch section that is provided between the detection section and a power supply node, and that changes a voltage of the detection section to a predetermined bias voltage by supplying a current from the power supply node to the detection section in an on state, and that cuts off the current from the power supply node to the detection section in an off state; a first circuit unit that generates a first detection signal based on the voltage change of the detection unit and a first periodic signal; a second circuit unit that generates a second detection signal based on the voltage change of the detection unit and a second periodic signal having a predetermined phase difference with respect to the first periodic signal; A counting unit that counts the first detection signal or the second detection signal and outputs a count value; a control unit that generates a first signal for causing the first circuit unit to generate the first detection signal and a second signal for causing the second circuit unit to generate the second detection signal; Equipped with the first periodic signal and the second periodic signal are generated based on the first signal or the second signal. Photoelectric conversion device. (Configuration 2) 2. The photoelectric conversion device according to configuration 1, wherein the first signal and the second signal are a common signal. (Configuration 3) the first periodic signal has a first phase difference with respect to the common signal; the second periodic signal has a second phase difference with respect to the common signal that is smaller than the first phase difference; a period during which the second circuit unit generates the second detection signal based on the voltage change of the detection unit is shorter than a period during which the first circuit unit generates the first detection signal based on the voltage change of the detection unit; 3. The photoelectric conversion device according to configuration 2. (Configuration 4) the first circuit unit generates the first detection signal based on the voltage change of the detection unit during a period in which the switch unit is in the on state; the second circuit unit generates the second detection signal based on the voltage change of the detection unit during a period in which the switch unit is in the on state. 4. The photoelectric conversion device according to configuration 3. (Configuration 5) The photoelectric conversion device according to configuration 4, wherein the second circuit section does not generate the second detection signal based on the voltage change of the detection section during a portion of a period during which the switch section is in the off state. (Configuration 6) The photoelectric conversion device according to configuration 3, further comprising a selection unit that selects a count value based on the first detection signal or a count value based on the second detection signal, and outputs the selected count value as a pixel value. (Configuration 7) A determination unit that determines whether the count value exceeds a predetermined threshold value, When the count value exceeds a predetermined threshold, the determination unit outputs a signal to the selection unit for selecting the count value based on the second detection signal. 7. The photoelectric conversion device according to configuration 6. (Configuration 8) 8. The photoelectric conversion device according to configuration 7, wherein the determination section outputs the signal for selecting the count value to the selection section within a period of one frame. (Configuration 9) The photoelectric conversion device according to configuration 7 or 8, wherein, when the count value is less than a predetermined threshold, the determination unit outputs a signal to the selection unit for selecting the count value based on the first detection signal generated by the first circuit unit. (Configuration 10) The photoelectric conversion device according to configuration 9, wherein the determination unit outputs a signal for identifying the first circuit unit or the second circuit unit that generated the selected first detection signal or the selected second detection signal. (Configuration 11) 11. The photoelectric conversion device according to any one of configurations 1 to 10, wherein the switch section is switched between the on state and the off state based on the first periodic signal generated in the control section. (Configuration 12) the second periodic signal is a signal having a phase difference with respect to the first signal or the second signal generated in the control unit, a period during which the second circuit unit generates the second detection signal based on the voltage change of the detection unit is shorter than a period during which the first circuit unit generates the first detection signal based on the voltage change of the detection unit; 12. The photoelectric conversion device according to claim 11. (Configuration 13) 13. The photoelectric conversion device according to configuration 12, wherein the first circuit section generates the first detection signal based on the voltage change of the detection section over a period during which the switch section is in the on state and the off state. (Configuration 14) the first periodic signal and the second periodic signal are pulse signals; the pulse width and frequency of the first periodic signal are equal to the pulse width and frequency of the second periodic signal; 14. The photoelectric conversion device according to any one of configurations 1 to 13. (Configuration 15) 15. The photoelectric conversion device according to any one of configurations 1 to 14, wherein the detection unit includes a SPAD. (Configuration 16) The photoelectric conversion device according to any one of configurations 1 to 15, a signal processing device that processes an image signal output from the photoelectric conversion device; An optical detection system comprising: (Configuration 17) The optical detection system of claim 16, wherein the signal processing device generates a distance image representing distance information to an object based on the image signal. (Configuration 18) A mobile object, The photoelectric conversion device according to any one of configurations 1 to 15, a distance information acquisition unit that acquires distance information to an object from an image signal output from the photoelectric conversion device; a moving object control unit that controls the moving object based on the distance information; A mobile body comprising: (Method 1) A method for controlling a photoelectric conversion device including a detection unit and a switch unit provided between the detection unit and a power supply node, the method comprising the steps of: causing a voltage change in the detection unit in response to incidence of a photon; a step of changing a voltage of the detection unit to a predetermined bias voltage by turning on the switch unit and supplying a current from the power supply node to the detection unit; turning off the switch unit to cut off a current from the power supply node to the detection unit; generating a first detection signal based on the voltage change of the detection unit and a first periodic signal; generating a second detection signal based on the voltage change of the detection unit and a second periodic signal having a predetermined phase difference with respect to the first periodic signal; counting the first detection signal or the second detection signal and outputting a count value; generating a first signal for generating the first detection signal or a second signal for generating the second detection signal; generating the first periodic signal and the second periodic signal based on the first signal or the second signal; A method for controlling a photoelectric conversion device, comprising:
[0177] The present disclosure can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.
[0178] It should be noted that the above-mentioned embodiments are merely examples of the realization of the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be embodied in various forms without departing from its technical concept or main features. [Explanation of symbols]
[0179] 40 Register Block 50 Pixel section 100 Photoelectric conversion device 110 Imaging system (light detection system) 510 SPAD 520 Switch section 530 Circuit section 540 Circuit section 551 Counting Department 552 Counting Department 560 Selection 570 Judgment section
Claims
1. A detection unit that generates a voltage change in response to the incidence of a photon; a switch section that is provided between the detection section and a power supply node, and that changes a voltage of the detection section to a predetermined bias voltage by supplying a current from the power supply node to the detection section in an on state, and that cuts off the current from the power supply node to the detection section in an off state; a first circuit unit that generates a first detection signal based on the voltage change of the detection unit and a first periodic signal; a second circuit unit that generates a second detection signal based on the voltage change of the detection unit and a second periodic signal having a predetermined phase difference with respect to the first periodic signal; a counting unit that counts the first detection signal or the second detection signal and outputs a count value; a control unit that generates a first signal for causing the first circuit unit to generate the first detection signal and a second signal for causing the second circuit unit to generate the second detection signal; Equipped with the first periodic signal and the second periodic signal are generated based on the first signal or the second signal. Photoelectric conversion device.
2. The photoelectric conversion device according to claim 1 , wherein the first signal and the second signal are a common signal.
3. the first periodic signal has a first phase difference with respect to the common signal; the second periodic signal has a second phase difference with respect to the common signal that is smaller than the first phase difference; a period during which the second circuit unit generates the second detection signal based on the voltage change of the detection unit is shorter than a period during which the first circuit unit generates the first detection signal based on the voltage change of the detection unit; The photoelectric conversion device according to claim 2 .
4. the first circuit unit generates the first detection signal based on the voltage change of the detection unit during a period in which the switch unit is in the on state; The photoelectric conversion device according to claim 3 , wherein the second circuit portion generates the second detection signal based on the voltage change of the detection portion during a period in which the switch portion is in the on state.
5. The photoelectric conversion device according to claim 4 , wherein the second circuit section does not generate the second detection signal based on the voltage change of the detection section during a part of a period during which the switch section is in the off state.
6. The photoelectric conversion device according to claim 3 , further comprising a selection section that selects a count value based on the first detection signal or a count value based on the second detection signal, and outputs the selected count value as a pixel value.
7. A determination unit that determines whether the count value exceeds a predetermined threshold value, When the count value exceeds a predetermined threshold, the determination unit outputs a signal to the selection unit for selecting the count value based on the second detection signal. The photoelectric conversion device according to claim 6 .
8. The photoelectric conversion device according to claim 7 , wherein the determination section outputs the signal for selecting the count value to the selection section within a period of one frame.
9. The photoelectric conversion device according to claim 7 , wherein when the count value is less than a predetermined threshold, the determination unit outputs a signal to the selection unit for selecting the count value based on the first detection signal generated by the first circuit unit.
10. The photoelectric conversion device according to claim 9 , wherein the determination unit outputs a signal for identifying the first circuit unit or the second circuit unit that generated the selected first detection signal or the selected second detection signal.
11. The photoelectric conversion device according to claim 1 , wherein the switch section is switched between the on state and the off state based on the first periodic signal generated in the control section.
12. the second periodic signal is a signal having a phase difference with respect to the first signal or the second signal generated in the control unit, a period during which the second circuit unit generates the second detection signal based on the voltage change of the detection unit is shorter than a period during which the first circuit unit generates the first detection signal based on the voltage change of the detection unit; The photoelectric conversion device according to claim 11 .
13. The photoelectric conversion device according to claim 12 , wherein the first circuit portion generates the first detection signal based on the voltage change of the detection portion over a period during which the switch portion is in the on state and the off state.
14. the first periodic signal and the second periodic signal are pulse signals, 2. The photoelectric conversion device according to claim 1, wherein a pulse width and a frequency of the first periodic signal are equal to a pulse width and a frequency of the second periodic signal.
15. The photoelectric conversion device according to claim 1 , wherein the detection unit includes a SPAD.
16. The photoelectric conversion device according to any one of claims 1 to 15, a signal processing device that processes an image signal output from the photoelectric conversion device; An optical detection system comprising:
17. The optical detection system according to claim 16 , wherein the signal processing device generates a distance image representing distance information to an object based on the image signal.
18. A mobile object, The photoelectric conversion device according to any one of claims 1 to 15, a distance information acquisition unit that acquires distance information to an object from an image signal output from the photoelectric conversion device; a moving object control unit that controls the moving object based on the distance information; A mobile body comprising:
19. A method for controlling a photoelectric conversion device including a detection unit and a switch unit provided between the detection unit and a power supply node, the method comprising the steps of: causing a voltage change in the detection unit in response to incidence of a photon; a step of changing a voltage of the detection unit to a predetermined bias voltage by turning on the switch unit and supplying a current from the power supply node to the detection unit; turning off the switch unit to cut off a current from the power supply node to the detection unit; generating a first detection signal based on the voltage change of the detection unit and a first periodic signal; generating a second detection signal based on the voltage change of the detection unit and a second periodic signal having a predetermined phase difference with respect to the first periodic signal; counting the first detection signal or the second detection signal and outputting a count value; generating a first signal for generating the first detection signal or a second signal for generating the second detection signal; generating the first periodic signal and the second periodic signal based on the first signal or the second signal; A method for controlling a photoelectric conversion device, comprising:
Citation Information
Patent Citations
Photoelectric conversion device and method for driving the same
JP2022096472A
Cited By
Game machine
JP2024166283A
Game machine
JP2024166284A
Game machine
JP2024166285A
Game machine
JP2024169533A
Game machine
JP2024169535A
