Photoelectric conversion device and photodetection system
The photoelectric conversion device addresses flicker issues by resetting and restarting the counting operation when certain threshold values are exceeded, enhancing performance and reducing power consumption.
Patent Information
- Application Number
- JP2023071302
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-25
- Publication Date
- 2024-11-07
AI Technical Summary
The photoelectric conversion device described in Patent Document 1 experiences flicker phenomena when photographing light sources that periodically turn on and off, such as fluorescent lamps or LED traffic lights.
A photoelectric conversion device with a counting control section that resets the count value and restarts the counting operation when the count value exceeds a second threshold, which is greater than a first threshold, to suppress flicker occurrence.
Effectively suppresses flicker occurrence while expanding the dynamic range and reducing power consumption.
Smart Images

Figure 2024157144000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a photoelectric conversion device and a photodetection system. [Background technology]
[0002] Single photon avalanche diodes (SPADs) are known as detectors capable of detecting weak light at the single photon level. SPADs amplify the signal charge excited by photons by several to several million times by using the avalanche multiplication phenomenon that occurs due to a strong electric field induced in the PN junction of a semiconductor. By converting the current generated by the avalanche multiplication phenomenon into a pulse signal and counting the number of pulse signals, it becomes possible to directly measure the number of incident photons.
[0003] Patent Document 1 describes a photoelectric conversion device that combines a recharge method that periodically recharges the SPAD with a method that performs counting operations for each pixel with an accumulation time according to the brightness of the subject. The photoelectric conversion device described in Patent Document 1 can achieve an expansion of the dynamic range and low power consumption. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2023-39400 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the photoelectric conversion device described in Patent Document 1, a flicker phenomenon may occur when capturing an image of a light source that periodically turns on and off, such as a fluorescent lamp or an LED traffic light.
[0006] An object of the present invention is to provide a photoelectric conversion device capable of effectively suppressing the occurrence of flicker. [Means for solving the problem]
[0007] According to one disclosure of the present specification, there is provided a photoelectric conversion device comprising: a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counting unit that counts the photon detection signal output from the photoelectric conversion unit; and a counting control unit that controls a counting operation in the counting unit, wherein the counting control unit resets the count value and restarts the counting operation from an initial value when a count value from a start of the counting operation to a first time is less than a first threshold value and a count value from a start of the counting operation to a second time that is later than the first time is equal to or greater than a second threshold value that is greater than the first threshold value.
[0008] According to another disclosure of the present specification, there is provided a photoelectric conversion device comprising: a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counting unit that counts the photon detection signal output from the photoelectric conversion unit; and a counting control unit that controls a counting operation in the counting unit, wherein the counting control unit has a first memory that holds a result of comparing a count value from the start of the counting operation to a first time with a first threshold value, a second memory that holds a result of comparing a count value from the start of the counting operation to a second time that is later than the first time with the first threshold value, and a third memory that holds a result of comparing the count value from the start of the counting operation to the second time with a second threshold value that is greater than the first threshold value, and the counting control unit controls the counting operation based on information held in the first memory, the second memory, and the third memory. Effect of the Invention
[0009] According to the present invention, the occurrence of flicker can be effectively suppressed. [Brief description of the drawings]
[0010] [Figure 1]1 is a block diagram showing a schematic configuration of a photoelectric conversion system according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a schematic configuration of a photoelectric conversion device according to a first embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of the configuration of a pixel in a photoelectric conversion device according to a first embodiment of the present invention. [Figure 4] 3 is a block diagram showing an example of the configuration of a counting control unit in the photoelectric conversion device according to the first embodiment of the present invention. FIG. [Diagram 5] 3 is a block diagram showing an example of the configuration of a comparison unit in the photoelectric conversion device according to the first embodiment of the present invention. FIG. [Figure 6] 4 is a timing chart showing the operations of a photoelectric conversion section and a counter section in the photoelectric conversion device according to the first embodiment of the present invention. FIG. [Figure 7] 11 is a graph showing the relationship between a count value and an accumulation time. [Figure 8] FIG. 4 is a diagram showing a criterion for determining whether or not recounting is necessary in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 9] FIG. 4 is a timing chart showing a recount operation in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 10] FIG. 1 is a timing chart showing the operation of a photoelectric conversion device using a PWEC system. [Figure 11] FIG. 4 is a timing chart showing the effect of suppressing flicker in the photoelectric conversion device according to the first embodiment of the present invention. [Figure 12] FIG. 13 is a block diagram showing an example of the configuration of a comparison unit in a photoelectric conversion device according to a second embodiment of the present invention. [Figure 13] FIG. 11 is a block diagram showing a schematic configuration of a light detection system according to a third embodiment of the present invention. [Figure 14] FIG. 13 is a block diagram showing a schematic configuration of a range image sensor according to a fourth embodiment of the present invention. [Figure 15] FIG. 13 is a schematic diagram showing an example of the configuration of an endoscopic surgery system according to a fifth embodiment of the present invention. [Figure 16]FIG. 13 is a schematic diagram showing a configuration example of a moving body according to a sixth embodiment of the present invention. [Figure 17] FIG. 13 is a block diagram showing a schematic configuration of a light detection system according to a sixth embodiment of the present invention. [Figure 18] FIG. 13 is a flow chart showing the operation of the light detection system according to the sixth embodiment of the present invention. [Figure 19] FIG. 13 is a schematic diagram showing a schematic configuration of a light detection system according to a seventh embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] The following embodiments are intended to embody the technical ideas of the present invention, and are not intended to limit the present invention. The sizes and positional relationships of the components shown in the drawings may be exaggerated to clarify the description. In the following description, the same components may be designated by the same reference numerals, and the description may be omitted.
[0012] [First embodiment] A photoelectric conversion system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 11. FIG. 1 is a block diagram showing a schematic configuration of a photoelectric conversion system according to this embodiment. FIG. 2 is a block diagram showing a schematic configuration of a photoelectric conversion device in the photoelectric conversion system according to this embodiment. FIG. 3 is a block diagram showing a configuration example of a pixel in the photoelectric conversion device according to this embodiment. FIG. 4 is a block diagram showing a configuration example of a counting control unit in the photoelectric conversion device according to this embodiment. FIG. 5 is a block diagram showing a configuration example of a comparison unit in the photoelectric conversion device according to this embodiment. FIG. 6 is a timing diagram showing the operation of the photoelectric conversion unit and the counting unit in the photoelectric conversion device according to this embodiment. FIG. 7 is a graph showing the relationship between the count value and the accumulation time. FIG. 8 is a diagram showing a criterion for determining whether or not recounting is necessary in the photoelectric conversion device according to this embodiment. FIG. 9 is a timing diagram showing the operation of recounting in the photoelectric conversion device according to this embodiment. FIG. 10 is a timing diagram showing the operation of the photoelectric conversion device using the PWEC method. FIG. 11 is a timing diagram showing the effect of flicker suppression in the photoelectric conversion device according to this embodiment.
[0013] First, the overall configuration of the photoelectric conversion system according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the photoelectric conversion system 100 according to this embodiment includes an optical lens 110, a photoelectric conversion device 120, a development processing unit 130, an external interface (I / F) unit 140, a storage unit 150, and an imaging control unit 160. The optical lens 110 is arranged so as to focus incident light on the photoelectric conversion device 120. The photoelectric conversion device 120 is connected to the development processing unit 130 and the imaging control unit 160. The development processing unit is connected to the external I / F unit 140, the storage unit 150, and the imaging control unit 160. The external I / F unit 140 is connected to the imaging control unit 160. The storage unit 150 is connected to the imaging control unit 160.
[0014] The optical lens 110 focuses the incident light on the imaging surface of the photoelectric conversion device 120. The photoelectric conversion device 120 converts the incident light into an electric signal and outputs it to the development processing unit 130 according to the operation setting from the imaging control unit 160. The development processing unit 130 performs demosaic processing on the electric signal received from the photoelectric conversion device 120, and also performs protocol conversion of the demosaic processed signal into a format that can be received by the external I / F unit 140. The storage unit 150 is a frame memory that temporarily stores output data of the development processing unit 130 in order to perform frame rate conversion and the like. The imaging control unit 160 has a role of controlling the operation of each part by setting operation parameters of the photoelectric conversion device 120, the development processing unit 130, the external I / F unit 140, and the storage unit 150, and the like.
[0015] Next, the photoelectric conversion device 120 will be described in more detail with reference to Fig. 2. As shown in Fig. 2, the photoelectric conversion device 120 has a pixel region 10, a register block 20, a vertical scanning circuit 40, a horizontal scanning circuit 50, a signal processing circuit 60, an output circuit 70, a control circuit 80, and a drive control circuit 90.
[0016] The pixel region 10 is provided with a plurality of pixels 12 arranged in an array having a plurality of rows and a plurality of columns. The number of pixels 12 constituting the pixel region 10 is not particularly limited. For example, the pixel region 10 may be constituted by a plurality of pixels 12 arranged in an array of several thousand rows and several thousand columns, as in a general digital camera. Alternatively, the pixel region 10 may be constituted by a plurality of pixels 12 arranged in one row or one column. Alternatively, the pixel region 10 may be constituted by a single pixel 12.
[0017] In each row of the pixel array in the pixel region 10, a control line 14 is arranged extending in a first direction (horizontal direction in FIG. 1). The control line 14 is connected to each of the pixels 12 arranged in the first direction, and serves as a signal line common to these pixels 12. The first direction in which the control lines 14 extend may be referred to as the row direction or horizontal direction. Each of the control lines 14 may include a plurality of signal lines for supplying a plurality of types of control signals to the pixels 12. The control lines 14 in each row are connected to a vertical scanning circuit 40.
[0018] In each column of the pixel array in the pixel region 10, a control line 16 is arranged to extend in a second direction (vertical direction in FIG. 1) intersecting the first direction. The control line 16 is connected to each of the pixels 12 aligned in the second direction, and serves as a signal line common to these pixels 12. The second direction in which the control lines 16 extend may be referred to as the column direction or vertical direction. Each of the control lines 16 may include a plurality of signal lines for supplying a plurality of types of control signals to the pixels 12. The control lines 16 in each column are connected to a horizontal scanning circuit 50.
[0019] Further, a periodic signal line 18, a setting signal line 22, and a pixel signal output line 24 are connected to each of the plurality of pixels 12 arranged in the pixel region 10. The periodic signal line 18 is connected to a drive control circuit 90. The setting signal line 22 is connected to a register block 20. The setting signal line 22 may include a plurality of signal lines for supplying a plurality of types of signals to the pixels 12. The pixel signal output line 24 may include a plurality of signal lines for transferring a multi-bit digital signal outputted from the pixel 12 on a bit-by-bit basis. The pixel signal output line 24 may be configured by a plurality of signal lines arranged in each column of the pixel array of the pixel region 10, or may be configured by a plurality of signal lines arranged in each row of the pixel array of the pixel region 10. The pixel signal output line 24 is connected to a signal processing circuit 60 and a control circuit 80.
[0020] The vertical scanning circuit 40 is a control circuit having a function of receiving a control signal output from the drive control circuit 90, generating a row control signal for driving the pixels 12, and supplying the row control signal to the pixels 12 via the control line 14. The vertical scanning circuit 40 may include logic circuits such as a shift register and an address decoder. The vertical scanning circuit 40 drives the pixels 12 in the pixel region 10 on a row-by-row basis by supplying a row control signal via the control line 14. The vertical scanning circuit 40 mainly has a role of sequentially supplying row control signals to the control line 14 of each row of the pixel region 10, and sequentially selecting a row for outputting a pixel signal. The vertical scanning circuit 40 also has a role of supplying a signal for internally driving the pixels 12 to the pixels 12.
[0021] The horizontal scanning circuit 50 is a control circuit having a function of receiving a control signal output from the drive control circuit 90, generating a column control signal for driving the pixels 12, and supplying the signal to the pixels 12 via the control line 16. The horizontal scanning circuit 50 may include logic circuits such as a shift register and an address decoder. The horizontal scanning circuit 50 drives the pixels 12 in the pixel region 10 on a column-by-column basis by supplying a column control signal via the control line 16. The horizontal scanning circuit 50 mainly has a role of sequentially supplying a column control signal to the control line 16 of each column of the pixel region 10, and sequentially selecting a column for outputting a pixel signal.
[0022] The register block 20 receives data from the imaging control unit 160 and a system disposed in a subsequent stage, such as an ISP (Image Signal Processor), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). The register block 20 has a role of holding the received data as setting values for various sensor operations of the photoelectric conversion device 120. The data communication means between the subsequent stage system and the register block 20 is not particularly limited, and for example, a serial communication means such as an I2C (Inter-Integrated Circuit) or an SPI (Serial Peripheral Interface) can be applied. The register block 20 stores all settings, such as the operation timing setting of the photoelectric conversion device 120, the pixel driving conditions, and the drive capacity setting in the analog driving part, and distributes them to each circuit via a setting signal line 22. For simplicity, FIG. 2 shows only the setting signal line 22 connecting the register block 20 and the pixel 12.
[0023] A pixel signal output line 24 is connected to each pixel 12, and the output signal of a pixel 12 selected by a row control signal from the vertical scanning circuit 40 and a column control signal from the horizontal scanning circuit 50 is output to a signal processing circuit 60 and a control circuit 80 via the pixel signal output line 24.
[0024] The signal processing circuit 60 is a processing circuit that performs predetermined digital arithmetic processing, such as offset processing and digital gain processing, on the pixel signals input via the pixel signal output lines 24 in response to a drive signal from the imaging control unit 160. Note that the digital arithmetic processing performed by the signal processing circuit 60 is not particularly limited, and various other digital arithmetic processing can be applied.
[0025] The output circuit 70 has a serializer that converts parallel data into serial data and an external interface circuit, and is a circuit for transmitting pixel signals after digital arithmetic processing in the signal processing circuit 60 to a downstream system such as an ISP, an ASIC, or an FPGA. As the external interface circuit, for example, a SerDes (SERializer / DESerializer) transmission circuit such as an LVDS (Low Voltage Differential Signaling) circuit or an SLVS (Scalable Low Voltage Signaling) circuit can be applied. Note that the configuration of the output circuit 70 is not limited to this, and other configurations such as parallel output may be used.
[0026] The control circuit 80 is connected to the register block 20, the signal processing circuit 60, and the output circuit 70. The control circuit 80 performs processing based on pixel signals input from the pixel signal output lines 24, and has a function of outputting signals to the signal processing circuit 60 and the output circuit 70 for limiting functions and reducing power consumption as necessary. The control circuit 80 also has a function of generating data for setting an operation mode of the pixel 12 based on the pixel signals input from the pixel signal output lines 24, and supplying the data to the register block 20.
[0027] The drive control circuit 90 is a control circuit for generating control signals for controlling the operation and timing of the vertical scanning circuit 40, the horizontal scanning circuit 50, etc., and supplying them to each circuit block. Although not shown in FIG. 1 for simplification, the drive control circuit 90 also simultaneously supplies control signals for controlling other circuit blocks other than the vertical scanning circuit 40 and the horizontal scanning circuit 50, and has a role of controlling the overall operation of the photoelectric conversion device 120. The drive control circuit 90 also supplies a periodic signal (clock signal CLK) to each of the multiple pixels 12 via a periodic signal line 18. The drive control circuit 90 may be driven by a clock signal input from the outside, or may be driven by a clock signal from a PLL circuit or the like provided inside the photoelectric conversion device 120. The drive control circuit 90 may also be driven by a drive signal supplied from the imaging control unit 160.
[0028] 3, each pixel 12 has a photoelectric conversion unit 30, a counting unit 32, a count control unit 34, a pixel data storage unit 36, and a pixel readout unit 38. The photoelectric conversion unit 30 has a photoelectric conversion element PD, a clock gate unit LG1, a waveform shaping unit LG2, and a switch unit MP.
[0029] The photoelectric conversion element PD may be configured with an avalanche photodiode (hereinafter, referred to as "APD"). The switch unit MP may be configured with a P-type MOS transistor. The anode of the APD that constitutes the photoelectric conversion element PD is connected to a node to which a voltage VL is supplied. The cathode of the APD that constitutes the photoelectric conversion element PD is connected to the drain of the P-type MOS transistor that constitutes the switch unit MP. The source of the P-type MOS transistor that constitutes the switch unit MP is connected to a node to which a voltage VH is supplied.
[0030] The clock gate unit LG1 may be configured, for example, by a two-input OR circuit. The waveform shaping unit LG2 may be configured, for example, by a two-input AND circuit. A signal (clock signal) CLK including a pulse with a predetermined period is input to one input node of the OR circuit constituting the clock gate unit LG1, and a signal (a stop signal to be described later) supplied from the counting control unit 34 is input to the other input node. The output node of the OR circuit constituting the clock gate unit LG1 is connected to the gate of the P-type MOS transistor constituting the switch unit MP. A periodic signal (clock signal CLK) is input to one input node of the AND circuit constituting the waveform shaping unit LG2, and an inverted signal of node A, which is a connection between the photoelectric conversion element PD and the switch unit MP, is input to the other input node. Node B, which is an output node of the AND circuit constituting the waveform shaping unit LG2, is connected to the counting unit 32.
[0031] The counting unit 32 is connected to a counting control unit 34 and a pixel data storage unit 36. The counting control unit 34 is connected to the pixel data storage unit 36. The pixel data storage unit 36 is connected to a pixel readout unit 38. The pixel readout unit 38 is connected to 24. An enable signal EN is input to the counting unit 32, and a reset signal RESET and a predetermined timing signal TIMING are input to the counting control unit. A control signal WRITE is input to the pixel data storage unit, and a control signal READ is input to the pixel readout unit 38.
[0032] The photoelectric conversion element PD may be constituted by an APD as described above. The voltages VL and VH are set so that a reverse bias voltage sufficient for the APD to perform avalanche multiplication is applied. In one example, a negative high voltage is applied as the voltage VL, and a positive voltage equivalent to the power supply voltage is applied as the voltage VH. For example, the voltage VL is about -30V, and the voltage VH is about 1V. By supplying a reverse bias voltage sufficient for avalanche multiplication to the APD, the charge generated by the incidence of light on the APD causes avalanche multiplication, and an avalanche current is generated. There are two operation modes when a reverse bias voltage is supplied to the APD: a Geiger mode and a linear mode. The Geiger mode is an operation mode in which the voltage applied between the anode and the cathode is a reverse bias voltage greater than the breakdown voltage of the APD. The linear mode is an operation mode in which the voltage applied between the anode and the cathode is a reverse bias voltage close to or less than the breakdown voltage of the APD. An APD operated in the Geiger mode is called a SPAD (Single Photon Avalanche Diode). The APD constituting the photoelectric conversion element PD may be operated in either the linear mode or the Geiger mode.
[0033] The switch unit MP has a role of switching between a period (photon detection period) for detecting photons incident on the photoelectric conversion element PD and a period (charging period) for charging the photoelectric conversion element PD. That is, when the switch unit MP is turned on upon receiving a low-level signal from the clock gate unit LG1, the voltage VH is supplied to the node A and the photoelectric conversion element PD is charged. When the switch unit MP is turned off upon receiving a high-level signal from the clock gate unit LG1, the node A is separated from the voltage VH and the potential of the node A is in a state in which it can change in response to the incidence of photons. The output signal of the clock gate unit LG1 is at low level when both the clock signal CLK and the signal (stop signal) from the counting control unit 34 are at low level, and is at high level otherwise.
[0034] The clock gate unit LG1 serves as a control circuit that periodically controls the APD between a standby state in which avalanche multiplication is possible and a recharge state in which the APD returns to a state in which avalanche multiplication is possible again after avalanche multiplication has occurred, in response to the clock signal CLK.
[0035] The waveform shaping unit LG2 has a role of outputting a photon detection signal indicating that a photon has been incident on the photoelectric conversion element PD during the photon detection period. That is, the waveform shaping unit LG2 outputs a high-level signal (photon detection signal) when the potential of node A becomes lower than a predetermined judgment threshold in response to the incidence of a photon on the photoelectric conversion element PD during the period (photon detection period) when the clock signal CLK is at a high level. The signal level of the output node (node B) of the waveform shaping unit LG2 becomes low at the timing when the photon detection period ends, that is, at the falling edge when the clock signal CLK next transitions from high level to low level.
[0036] The counting unit 32 has a counter that counts up every time a rising edge of the photon detection signal output from the waveform shaping unit LG2 is detected, and outputs a count value that is the counting result of the counter to the count control unit 34 and the pixel data storage unit 36. The counting operation in the counting unit 32 is performed during a period (accumulation period) when the enable signal EN is at a high level. The count value in the counting unit 32 is reset in response to a reset instruction from the count control unit 34.
[0037] The pixel data storage unit 36 stores a pixel signal based on the count value of the counter 32 and the output signal of the count control unit 34 in response to a write instruction (control signal WRITE) supplied from the control circuit 80 via the setting signal line 22.
[0038] The pixel read unit 38 outputs the pixel signal stored in the pixel data storage unit 36 to the pixel signal output line 24 in response to a read instruction (control signal READ) supplied from the control circuit 80 via the setting signal line 22.
[0039] The count control unit 34 has a role of controlling the counting operation in the counting unit 32 in response to the count value in the counting unit 32, a predetermined timing signal TIMING, and a reset signal RESET. The control of the counting operation includes judging whether to continue, stop, or restart the counting operation and executing the same.
[0040] Next, a specific configuration example of the count control unit 34 will be described with reference to FIG. 4. The count control unit 34 may be configured with comparison units 341 and 342, storage units 343 and 344, a recount determination unit 345, a stop signal generation unit 346, and a reset signal generation unit 347, as shown in FIG. 4, for example. The comparison unit 341 has latch circuits LC1 and LC2. The comparison unit 342 has a latch circuit LC3. The storage unit 343 has memories MEM0, MEM1, and MEM2. The storage unit 344 has memories MEM4 and MEM5. The recount determination unit 345 has two-input XOR circuits LG3 and LG4. The stop signal generation unit 346 has a four-input OR circuit LG5. The reset signal generation unit 347 has two-input AND circuits LG6 and LG7 and a three-input OR circuit LG8.
[0041] The count value of the counter 32 is input to the comparators 341 and 342. The latch circuits LC1 and LC2 hold a signal corresponding to the result of the comparison in the comparator 341. The latch circuit LC3 holds a signal corresponding to the result of the comparison in the comparator 342. The output node of the latch circuit LC1 is connected to the input nodes of the memories MEM0, MEM1, and MEM2. The output node of the latch circuit LC2 is connected to the input node of the OR circuit LG5. The output node of the latch circuit LC3 is connected to the input nodes of the memories MEM3 and MEM4 and the input nodes of the AND circuits LG6 and LG7. The output node of the memory MEM0 is connected to the pixel data storage unit 36 and the input node of the OR circuit LG5. The output node of the memory MEM1 is connected to the input node of the XOR circuit LG3. The output node of the memory MEM2 is connected to the input node of the XOR circuit LG4. The output node of the memory MEM3 is connected to the input node of the XOR circuit LG3 and the input node of the AND circuit LG6. The output node of the memory MEM4 is connected to the input node of the XOR circuit LG4 and the input node of the AND circuit LG7.
[0042] An output node of the XOR circuit LG3 is connected to the pixel data storage unit 36 and an input node of the OR circuit LG5. An output node of the XOR circuit LG4 is connected to the pixel data storage unit 36 and an input node of the OR circuit LG5. An output node of the OR circuit LG5 is connected to an input node of the clock gate unit LG1 and the counter unit 32. An output node of the AND circuit LG6 and an output node of the AND circuit LG7 are connected to an input node of the OR circuit LG8. A reset signal RESET is supplied to the other input node of the OR circuit LG8. An output node of the OR circuit LG8 is connected to the counter unit 32 and the comparators 341, 342.
[0043] The comparator 341 compares the count value acquired from the counter 32 with a first threshold value, and causes the latch circuit LC1 to hold a high-level signal if the acquired count value is equal to or greater than the first threshold value. Here, the first threshold value is a threshold value for determining pixel saturation. For example, if the counter of the counter 32 is configured as an 11-bit counter and the count saturation value (Nsat) is 2048 LSB, the first threshold value can be set to 256 LSB (=Nsat / 8). In this case, the latch circuit LC1 can be configured as a D latch that receives the 8th bit of the count value supplied from the counter 32 as an input. Note that the first threshold value is set to 2 N It is preferable to set the first threshold to 2 N By setting the count value to Nth bit, the function can be realized with a small circuit that simply outputs the Nth bit of the count value to the latch circuit LC3.
[0044] Furthermore, the comparator 341 compares the count value acquired from the counter 32 with the upper count limit of the counter 32, and causes the latch circuit LC2 to hold a high-level signal when the acquired count value is the upper count limit of the counter 32. For example, when the counter of the counter 32 is configured as an 11-bit counter, the upper count limit is 2047 LSB (=Nsat-1). In this case, the latch circuit LC2 can be configured as a D latch that receives the 11th bit of the count value supplied from the counter 32 as an input.
[0045] The comparison unit 342 compares the count value acquired from the counting unit 32 with the second threshold, and when the acquired count value is equal to or greater than the second threshold, causes the latch circuit LC3 to hold a high-level signal. Here, the second threshold is a threshold used to determine whether or not recounting is required, and is set to a value greater than the first threshold. For example, when the counter of the counting unit 32 is configured as an 11-bit counter and the first threshold is 256 LSB, the second threshold (Nth) can be set to 1536 LSB. In this case, the comparison unit 342 can be configured, for example, as shown in FIG. 5, with a D latch D1, a D latch D2, and a two-input AND circuit LG9. The D latch D1 receives the 10th bit (Count[9]) of the count value at its D terminal and the enable signal Enable at its E terminal as inputs, respectively. The D latch D2 receives the 9th bit (Count[8]) of the count value at its D terminal and the enable signal Enable at its E terminal as inputs, respectively. The AND circuit LG9 receives as input the output of the D latch D1 and the output of the D latch D2, and outputs the logical product of these. The output of the AND circuit LG9 becomes the output of the latch circuit LC3. The second threshold may be configured to be dynamically changeable. The enable signal Enable may be a periodic signal (clock signal CLK) input to the photoelectric conversion unit 30, or may be an enable signal EN input to the counting unit 32.
[0046] The memories MEM0, MEM1, and MEM2 of the storage unit 343 are configured to hold the output of the latch circuit LC1 in synchronization with the timing signal TIMING. Specifically, the memory MEM0 holds the output of the latch circuit LC1 at time T0, the memory MEM1 holds the output of the latch circuit LC1 at time T1, and the memory MEM2 holds the output of the latch circuit LC1 at time T2. Here, the times T0, T1, and T2 are times that arrive in this order during one accumulation time. The timing signal TIMING is a signal whose level changes at times T0, T1, and T2, and the memories MEM0, MEM1, and MEM2 hold the output of the latch circuit LC1 in synchronization with the edges of the timing signal TIMING.
[0047] The memories MEM3 and MEM4 of the storage unit 344 are configured to hold the output of the latch circuit LC3 in synchronization with the timing signal TIMING. Specifically, the memory MEM3 holds the output of the latch circuit LC3 at time T1, and the memory MEM4 holds the output of the latch circuit LC3 at time T2. The memories MEM3 and MEM4 hold the output of the latch circuit LC3 in synchronization with the edges of the timing signal TIMING, similar to the memories MEM0, MEM1, and MEM2.
[0048] The XOR circuit LG3 of the recount determination unit 345 is used to generate a control signal based on the result of the determination of whether or not a recount is necessary at time T1. The XOR circuit LG3 receives as input the output of the memory MEM1 and the output of the memory MEM3, and outputs an exclusive OR of these. The output signal of the XOR circuit LG3 is input to the pixel data storage unit 36 and the stop signal generation unit 346. The XOR circuit LG4 of the recount determination unit 345 is used to generate a control signal based on the result of the determination of whether or not a recount is necessary at time T2. The XOR circuit LG4 receives as input the output of the memory MEM2 and the output of the memory MEM4, and outputs an exclusive OR of these. The output signal of the XOR circuit LG4 is input to the pixel data storage unit 36 and the stop signal generation unit 346.
[0049] The output signal of the latch circuit LC2, the output signal of the memory MEM0, the output signal of the XOR circuit LG3, and the output signal of the XOR circuit LG4 are input to the input node of the OR circuit LG5 of the stop signal generating unit 346. The output signal of the OR circuit LG5 is input to the counting unit 32 and the clock gate unit LG1.
[0050] The output signal of the latch circuit LC3 and the output signal of the memory MEM3 are input to the input node of the AND circuit LG6 of the reset signal generating unit 347. The output signal of the AND circuit LG6 is input to the OR circuit LG8. The output signal of the latch circuit LC3 and the output signal of the memory MEM4 are input to the input node of the AND circuit LG7. The output signal of the AND circuit LG7 is input to the OR circuit LG8. The output signal of the OR circuit LG8 is input to the counting unit 32 and the comparison units 341, 342.
[0051] Next, an example of the operation of the photoelectric conversion device according to this embodiment will be described with reference to Figs. 6 to 9. In this embodiment, a method of driving the photoelectric conversion device that combines a clocked recharging method and a pixel-wise exposure control (PWEC) method will be described. Here, the recharging method is a method of periodically recharging the photoelectric conversion element PD. The PWEC method is a method of performing a counting operation for each pixel 12 with a normal accumulation time when the brightness is normal, and with a short accumulation time when the brightness is extremely high.
[0052] First, the basic operations of the photoelectric conversion unit 30 and the counting unit 32 will be described with reference to the timing diagram of Fig. 6. Fig. 6 shows the signal level of the clock signal CLK, the voltage of node A, the signal level of node B, and the count value of the counter of the counting unit 32. Fig. 6 also shows the timing at which photons enter the photoelectric conversion element PD with an arrow above the signal waveform.
[0053] Immediately before time t1, it is assumed that the clock signal CLK is at a high level, the voltage of node A is VH, and the signal level of node B is at a high level. Also, it is assumed that the count value of the counter of the counter unit 32 is N.
[0054] At time t1, the clock signal CLK transitions from high to low. This turns on the switch MP, and the charging operation of the photoelectric conversion element PD begins. After that, the clock signal CLK transitions from low to high, but the voltage VH at node A remains, and the signal level at node B remains low.
[0055] At time t2 after the clock signal CLK goes low and the photon detection period starts, a photon is incident on the photoelectric conversion element PD. This causes avalanche multiplication in the photoelectric conversion element PD, and the voltage of node A drops from voltage VH to a predetermined voltage according to the on-resistance of the P-type transistor. When the voltage of node A falls below a predetermined determination threshold, the signal level of node B, which is the output node of the waveform shaping unit LG2, transitions from low to high. This causes the count value of the counter of the counting unit 32 to increase by 1 LSB from N to N+1.
[0056] After time t2, photons are assumed to be incident on the photoelectric conversion element PD at times t3 and t4, which are prior to time t5, when the clock signal CLK next transitions from high level to low level. At times t3 and t4, avalanche multiplication has already occurred in the photoelectric conversion element PD, and the photoelectric conversion element PD is in a state prior to recharging. Therefore, the signal levels of the nodes do not transition due to the photons incident at times t3 and t4, and the photons incident at times t3 and t4 are not counted by the counter 32. Photons that are not counted in this manner are indicated by dashed arrows above the voltage waveform of node A.
[0057] At the next time t5, the clock signal CLK transitions from high to low. This turns on the switch MP, starts charging the photoelectric conversion element PD, and the voltage of node A returns to voltage VH. Then, in response to the voltage change of node A, the signal level of node B transitions from high to low.
[0058] At the next time t6, the clock signal CLK transitions from high to low. If no photons are incident on the photoelectric conversion element PD during the period from time t5 to time t6, the voltage at node A and the signal level at node B do not change during this period.
[0059] Similarly, after time t7, if a photon is incident during the period from when the clock signal CLK transitions from a low level to a high level until the next transition of the clock signal CLK from a high level to a low level, a photon detection signal is output from the waveform shaping unit LG2. For example, during the period from time t6 to time t8, the count value of the counter 32 increases by 1 LSB from N+1 to N+2 in response to the incidence of a photon at time t7. Also, during the period from time t8 to time t10, the count value of the counter 32 increases by 1 LSB from N+2 to N+3 in response to the incidence of a photon at time t9. On the other hand, if no photon is incident during the period from when the clock signal CLK transitions from a low level to a high level until the next transition of the clock signal CLK from a high level to a low level, the voltage of node A and the signal level of node B do not change. That is, photon counting is not performed.
[0060] That is, the counter 32 operates to count the number of periods during which at least one avalanche multiplication occurs in the photoelectric conversion element PD, within a period defined by the interval between pulses superimposed on the clock signal CLK.
[0061] Next, an example of the operation of the count control unit 34 will be described with reference to Figs. 7 to 9. Here, in order to provide a specific explanation of driving by the PWEC method, the following conditions are assumed. That is, the counting unit 32 is configured with an 11-bit counter, and the count saturation value (Nsat) is 2048. The driving speed of the pixel 12 is 60 frames per second (=16.6 ms / frame), and the accumulation times are 16.6 ms (T), 2.07 ms (T2=T / 8), and 259 µs (T1=T / 8). 2 ), 32.4μs(T0=T / 8 3), and Nsat / 8 LSB. A first threshold is set as a threshold for determining pixel saturation, and its value is 256 (Nsat / 8) LSB. The pixel saturation is determined at each of the times T0, T1, and T2 after the start of the accumulation period. A second threshold is set as a threshold for determining whether or not recounting is required, and its value is 1536 LSB. The recount is determined at each of the times T1 and T2 after the start of the accumulation period. However, these conditions change depending on the basic configuration of the system and the measurement environment, such as the number of bits of the counter constituting the counting unit 32, and the blinking cycle and brightness of the light source, and therefore the configuration of each unit of the pixel 12 can also be appropriately changed depending on these conditions.
[0062] The count control unit 34 acquires the count value held by the counting unit 32. The comparison units 341 and 342 compare the count value acquired from the counting unit 32 with a predetermined judgment value, and cause the latch circuits LC1, LC2, and LC3 to hold a judgment value according to the comparison result. That is, the comparison unit 341 compares the acquired count value with a first threshold value that is a criterion for saturation judgment, and causes the latch circuit LC1 to hold a high-level judgment signal when the count value is equal to or greater than the first threshold value. The comparison unit 341 also compares the acquired count value with the count upper limit value of the counting unit 32, and causes the latch circuit LC2 to hold a high-level judgment signal when the count value is equal to the count upper limit value of the counting unit 32. The comparison unit 342 compares the acquired count value with a second threshold value that is a criterion for recount judgment, and causes the latch circuit LC3 to hold a high-level judgment signal when the count value is equal to or greater than the second threshold value.
[0063] FIG. 7 is a graph showing the relationship between the count value of the counter 32 and the accumulation time in four pixels 12A, 12B, 12C, and 12D having different amounts of incident light. In FIG. 7, the pixel 12A has a count value of 32.4 μs (=T / 8 3 It is assumed that the count value at time T1 (accumulation time 259 μs (=T / 8)) is 2000 LSB. 2For pixel 12C, it is assumed that the count value at time T2 (accumulation time 2.07 ms (=T / 8)) is 500 LSB. For pixel 12C, it is assumed that the count value at time T2 (accumulation time 2.07 ms (=T / 8)) is 350 LSB. For pixel 12D, it is assumed that the count value at time T (accumulation time 16.6 ms (=T)) is 400 LSB.
[0064] The pixel saturation determination using the first threshold is performed at each of the timings T0, T1, and T2. For example, when the first threshold is 256 LSB (=Nsat / 8), the pixel 12A is determined to be saturated at the time T0, the pixel 12B is determined to be saturated at the time T1, and the pixel 12C is determined to be saturated at the time T2. The pixel 12D is not determined to be saturated at any of the times T0, T1, and T2. The results of the saturation determination using the first threshold at the times T0, T1, and T2 are stored in the memories MEM0, MEM1, and MEM2 of the storage unit 343, respectively. That is, when it is determined that there is pixel saturation, a high-level signal is stored in the corresponding memory, and when it is determined that there is no pixel saturation, a low-level signal is stored in the corresponding memory.
[0065] The determination of whether or not recounting is necessary using the second threshold is performed at each of times T1 and T2. The method of determining whether or not recounting is necessary is the same as the method of saturation determination, except that the determination threshold is different. The results of the determination of whether or not recounting is necessary at times T1 and T2 using the second threshold are stored in memories MEM3 and MEM4 of the storage unit 344, respectively. That is, if it is determined that there is a high need for recounting, a high-level signal is stored in the corresponding memory, and if it is determined that there is a low need for recounting, a low-level signal is stored in the corresponding memory.
[0066] In the PWEC method, the final count value is calculated by multiplying each count value by a coefficient according to the length of the accumulation time. For example, the final count value of pixel 12A is calculated by multiplying 2000LSB by a coefficient of 8. 3 The final count value of pixel 12B is 500LSB multiplied by a coefficient of 8, which is 1024000LSB. 2The final count value of pixel 12C is 350LSB multiplied by a coefficient of 8. 3 The final count value of pixel 12D is 400 LSB multiplied by the coefficient 1, resulting in 400 LSB.
[0067] When the count value acquired from the counter unit 32 is the upper count limit of the counter unit 32, the output of the latch circuit LC2 becomes high level, and the output of the OR circuit LG5 constituting the stop signal generating unit 346 becomes high level. As a result, the output of the OR circuit constituting the clock gate unit LG1 becomes high level, and recharging of the photoelectric conversion element PD and the counting operation in the counter unit 32 are stopped.
[0068] If it is determined that pixel saturation has occurred as a result of the saturation determination at time T0, the output of the memory MEM0 goes high, and the output of the OR circuit LG5 constituting the stop signal generating unit 346 goes high. This causes the output of the OR circuit constituting the clock gate unit LG1 to go high, stopping the recharging of the photoelectric conversion element PD and the counting operation in the counter unit 32. If it is determined that pixel saturation has not occurred as a result of the saturation determination at time T0, the process moves to saturation determination at times T1 and T2 and a determination as to whether or not recounting is required.
[0069] The recount determination unit 345 determines whether or not recounting is necessary based on the results of the saturation determination and the determination of whether or not recounting is necessary at times T1 and T2, i.e., based on the information stored in the memories MEM1, MEM2, MEM3, and MEM4. Fig. 8 summarizes the criteria for determining whether or not recounting is necessary in the recount determination unit 345. Fig. 8(a) shows the determination criteria at time T1, and Fig. 8(b) shows the determination criteria at time T2.
[0070] If it is determined at time T1 that the output of memory MEM1 and the output of memory MEM3 are both at a low level, then no re-counting is performed, and the counting operation in counting unit 32 continues. If the output of memory MEM1 is at a high level and the output of memory MEM3 is at a low level, pixel saturation has occurred, so the counting operation is stopped (no re-counting) as in the case of the saturation determination at time T0. If the output of memory MEM1 and the output of memory MEM3 are both at a high level, then re-counting is performed. Note that the condition in which the output of memory MEM1 is at a low level and the output of memory MEM3 is at a high level cannot occur.
[0071] Similarly, if it is determined at time T2 that the output of memory MEM2 and the output of memory MEM4 are both at a low level, no re-counting is performed, and the counting operation in counting unit 32 continues. If the output of memory MEM2 is at a high level and the output of memory MEM4 is at a low level, pixel saturation has occurred, so the counting operation is stopped (no re-counting) as in the case of the saturation determination at time T0. If the output of memory MEM2 and the output of memory MEM4 are both at a high level, re-counting is performed. Note that the condition in which the output of memory MEM2 is at a low level and the output of memory MEM4 is at a high level cannot occur.
[0072] The recounting decision is made for pixels 12 that are non-saturated (less than the first threshold) in the saturation decision at time T0, but whose count value increases suddenly at time T1 or time T2. For such pixels 12, there is a high possibility that accumulation has started when a blinking light source, such as an LED traffic light, is turned off, and there is a possibility that flicker will be induced when the image is visualized, so recounting is performed. Note that the impact of image defects and increased power consumption due to recounting is minor.
[0073] When the saturation determination and the re-count determination are made at time T1, the output of the latch circuit LC3 and the output of the memory MEM2 become high level, and the output of the AND circuit LG6 of the reset signal generation unit 347 becomes high level. As a result, the output of the OR circuit LG8 becomes high level, and the count value of the counting unit 32 and the values of the latch circuits LC1, LC2, and LC3 are reset regardless of the signal level of the reset signal RESET from the control circuit 80, and the counting unit 32 starts to re-count. Similarly, when the saturation determination and the re-count determination are made at time T2, the output of the latch circuit LC3 and the output of the memory MEM4 become high level, and the output of the AND circuit LG7 of the reset signal generation unit 347 becomes high level. As a result, the output of the OR circuit LG8 becomes high level, and the count value of the counting unit 32 and the values of the latch circuits LC1, LC2, and LC3 are reset regardless of the reset signal RESET from the control circuit 80, and the counting unit 32 starts to re-count.
[0074] In this embodiment, in order to prevent the counting operation from stopping when the saturation determination and the re-count determination are made simultaneously, XOR circuits LG3 and LG4 are inserted between the memories MEM1, MEM2, MEM3, and MEM4 and the OR circuit LG5. This makes it possible to stop the counting operation only when the output of only the memory MEM1 of the memories MEM1 and MEM3 becomes high level, or when the output of only the memory MEM2 of the memories MEM2 and MEM4 becomes high level.
[0075] Fig. 9 is a timing diagram when it is determined that re-counting is to be performed at time T1. Fig. 9 shows waveforms of the enable signal EN, the reset signal RESET, the output signal of the latch circuit LC3, the output signal of the memory MEM3, the output signal of the AND circuit LG6, and the output signal of the OR circuit LG8.
[0076] At time t21, the reset signal RESET goes high, the count value of the counter of the counting unit 32 is reset to the initial value 0, and the outputs of the latch circuits LC1, LC2, and LC3 are reset to low. At the following time t22, the enable signal EN goes high, and the counting operation of the photon detection signal in the counting unit 32 is started. At the following time t22, when the count value of the counter of the counting unit 32 reaches the second threshold, the output of the latch circuit LC3 goes high. Then, at time T1, the output of the memory MEM3 goes high, and the output of the AND circuit LG6 goes high. As a result, the output of the OR circuit LG8 goes high, and the latch circuit LC3 is reset at time t23. In response to this, the output of the AND circuit LG6 goes low at time t24, and the output of the OR circuit LG8 goes low at time t25. The period from time T1 when the output of the OR circuit LG8 goes high to time t25 is the reset period of the counting unit 32 for recounting. After the reset period has elapsed, the counting unit 32 starts counting again.
[0077] The signals output from memory MEM0 and XOR circuits LG3 and LG4 to pixel data storage unit 36 can be used as time information when calculating the final count value. That is, when the output signal of memory MEM0 is at a high level, it indicates that the counting operation stopped at time T0. When the output signal of XOR circuit LG3 is at a high level, it indicates that the counting operation stopped at time T1. When the output signal of XOR circuit LG4 is at a high level, it indicates that the counting operation stopped at time T2. Therefore, the final count value can be calculated by selecting a coefficient by which to multiply the count value output from counter 32 in accordance with the signals output from memory MEM0 and XOR circuits LG3 and LG4.
[0078] Next, the effect of suppressing flicker in the photoelectric conversion device according to this embodiment will be described with reference to Figs. 10 and 11. Fig. 10 is a timing diagram showing the operation of a typical photoelectric conversion device using the PWEC method, and Fig. 11 is a timing diagram showing the operation of the photoelectric conversion device according to this embodiment. Here, an LED traffic light is assumed as the blinking light source that causes flicker, and Figs. 10 and 11 also show the off period and blinking period of the light source. Here, it is assumed that the blinking cycle of the LED traffic light is 100 Hz and the duty ratio is 80% (repeated 2 ms off and 8 ms on).
[0079] First, the operation of a typical photoelectric conversion device using the PWEC method will be described. In Fig. 10, it is assumed that the accumulation period starts with the LED traffic light turned off (solid line in Fig. 10). When the LED traffic light is turned off, a relatively long accumulation time, for example T / 8 (=2.07 ms), is set and the counting operation is executed. The count value is low while the LED traffic light is turned off, but increases rapidly when the LED traffic light is turned on, reaching 2047 LSB at time T2, the end of the accumulation period. The actual count value in this case is 16387 LSB, which is 2047 LSB multiplied by 8.
[0080] On the other hand, if the accumulation period starts with the LED traffic light on (dotted line in Fig. 10), a relatively short accumulation time, for example T / 8 3 (=32.4μs) is set and the count operation is executed. If the count value reaches 2000LSB at time T0, the end time of the accumulation period, the actual count value is 8 times 2000LSB. 3 This results in 1,024,000 LSB multiplied by 1. In other words, the count value, or brightness, acquired depending on whether the LED traffic light is off or on at the start of the accumulation period will vary greatly, and will appear as flicker when visualized.
[0081] Next, the operation of the photoelectric conversion device according to this embodiment will be described. In FIG. 11, as in FIG. 10, it is assumed that the accumulation period starts with the LED traffic light turned off. When the LED traffic light is turned off, a relatively long accumulation time, for example, T / 8 (=2.07 ms), is set, and the counting operation is performed. The count value is a low value while the LED traffic light is turned off, and is below the first threshold value (Nsat / 8=256 LSB) at times T0 and T1. If the count value exceeds the second threshold value (Nth=1536 LSB) at time T2, a decision is made to count again, the first count value is discarded, and a new counting operation is started from time T2. This makes it possible to obtain an appropriate count value that is free from the effects of flicker.
[0082] Although the case where a recount determination is made at time T2 has been described here, the same applies to the case where a recount determination is made at time T1. Also, even if a recount determination is made again during the recount operation, it can be handled by repeating the same process as that performed before the recount operation.
[0083] As described above, according to this embodiment, it is possible to effectively suppress the occurrence of flicker while expanding the dynamic range and reducing power consumption.
[0084] [Second embodiment] A photoelectric conversion device according to a second embodiment of the present invention will be described with reference to Fig. 12. The same components as those in the photoelectric conversion device according to the first embodiment are given the same reference numerals, and descriptions thereof will be omitted or simplified. Fig. 12 is a block diagram showing an example of the configuration of a comparison unit in the photoelectric conversion device according to this embodiment.
[0085] The photoelectric conversion device according to this embodiment is similar to the photoelectric conversion device according to the first embodiment except for the configuration of the comparison unit 341. In this embodiment, the differences from the photoelectric conversion device of the first embodiment will be mainly described, and descriptions of the same parts as those of the photoelectric conversion device of the first embodiment will be omitted as appropriate.
[0086] In the first embodiment, the first threshold value used to determine pixel saturation is fixed to 256 LSB (=Nsat / 8). In contrast, in the present embodiment, the first threshold value used to determine pixel saturation can be selected from a plurality of preset threshold values. Note that, as an example, a configuration in which the first threshold value is selected from 256 LSB (=Nsat / 8), 512 LSB (=Nsat / 4), and 1024 LSB (Nsat / 2) will be described here, but the value and number of the setting values prepared as the first threshold value can be changed as appropriate.
[0087] As shown in Fig. 12, the comparison unit 341 of the photoelectric conversion device according to this embodiment is composed of latch circuits LC11, LC12, LC13, and LC2, and a multiplexer MUX. A count value is input from the counter 32 to each of the latch circuits LC11, LC12, LC13, and LC2. The latch circuits LC11, LC12, and LC13 are connected to the multiplexer MUX. The multiplexer MUX is connected to the storage unit 343. The latch circuit LC2 is connected to an OR circuit LG5.
[0088] The comparator 341 compares the count value acquired from the counter 32 with the thresholds set in the latch circuits LC11, LC12, and LC13, and when the acquired count value is equal to or greater than the set threshold, causes the corresponding latch circuit to hold a high-level signal. That is, when the count value is equal to or greater than 256 LSB (=Nsat / 8), a high-level signal is held in the latch circuit LC11. When the count value is equal to or greater than 512 LSB (=Nsat / 4), a high-level signal is held in the latch circuit LC12. When the count value is equal to or greater than 1024 LSB (=Nsat / 2), a high-level signal is held in the latch circuit LC13. The multiplexer MUX selects a latch circuit corresponding to the set value of the first threshold from among the latch circuits LC11, LC12, and LC13 in accordance with a control signal supplied from the control circuit 80 via the setting signal line 22, and outputs the signal held by the selected latch circuit to the memory unit 343.
[0089] By adopting the configuration of this embodiment, the circuit scale of the pixel 12 becomes larger than that of the first embodiment, but it is possible to increase the flexibility of the system.
[0090] As described above, according to this embodiment, it is possible to effectively suppress the occurrence of flicker while expanding the dynamic range and reducing power consumption.
[0091] [Third embodiment] A light detection system according to a third embodiment of the present invention will be described with reference to Fig. 13. Fig. 13 is a block diagram showing a schematic configuration of the light detection system according to this embodiment. In this embodiment, a light detection sensor to which the photoelectric conversion device 120 described in the first or second embodiment is applied will be described.
[0092] The photoelectric conversion device 120 described in the first and second embodiments is applicable to various photodetection systems. Examples of the applicable photodetection systems include imaging systems such as digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites. Camera modules equipped with an optical system such as a lens and an imaging device are also included in the photodetection system. FIG. 13 illustrates a block diagram of a digital still camera as an example of these.
[0093] 13 includes a photoelectric conversion device 201, a lens 202 that forms an optical image of a subject on the photoelectric conversion device 201, an aperture 204 that varies the amount of light passing through the lens 202, and a barrier 206 that protects the lens 202. The lens 202 and the aperture 204 form an optical system that focuses light on the photoelectric conversion device 201. The photoelectric conversion device 201 is the photoelectric conversion device 120 described in the first or second embodiment, and converts the optical image formed by the lens 202 into image data.
[0094] The light detection system 200 also includes a signal processing unit 208 that processes an output signal output from the photoelectric conversion device 201. The signal processing unit 208 generates image data from a digital signal output from the photoelectric conversion device 201. The signal processing unit 208 also performs various corrections and compression as necessary to output image data. The photoelectric conversion device 201 may include an AD conversion unit that generates a digital signal to be processed by the signal processing unit 208. The AD conversion unit may be formed in a semiconductor layer (semiconductor substrate) in which the photon detection element of the photoelectric conversion device 201 is formed, or may be formed in a semiconductor substrate different from the semiconductor layer in which the photon detection element of the photoelectric conversion device 201 is formed. The signal processing unit 208 may also be formed in the same semiconductor substrate as the photoelectric conversion device 201.
[0095] The light detection system 200 further includes a memory unit 210 for temporarily storing image data, and an external interface unit (external I / F unit) 212 for communicating with an external computer or the like. The light detection system 200 further includes a recording medium 214 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 216 for recording or reading out the imaging data on the recording medium 214. The recording medium 214 may be built into the light detection system 200, or may be removable. Furthermore, communication between the recording medium control I / F unit 216 and the recording medium 214 and communication from the external I / F unit 212 may be performed wirelessly.
[0096] The light detection system 200 further includes an overall control / calculation unit 218 that performs various calculations and controls the entire digital still camera, and a timing generation unit 220 that outputs various timing signals to the photoelectric conversion device 201 and the signal processing unit 208. Here, the timing signals and the like may be input from outside, and the light detection system 200 only needs to include at least the photoelectric conversion device 201 and the signal processing unit 208 that processes the output signal output from the photoelectric conversion device 201. The timing generation unit 220 may be mounted on the photoelectric conversion device 201. Moreover, the overall control / calculation unit 218 and the timing generation unit 220 may be configured to implement a part or all of the control functions of the photoelectric conversion device 201.
[0097] The photoelectric conversion device 201 outputs an imaging signal to the signal processing unit 208. The signal processing unit 208 performs predetermined signal processing on the imaging signal output from the photoelectric conversion device 201 and outputs image data. The signal processing unit 208 generates an image using the imaging signal. The signal processing unit 208 may be configured to perform distance measurement calculation on the signal output from the photoelectric conversion device 201.
[0098] As described above, according to this embodiment, by configuring a light detection system using the photoelectric conversion device of the first or second embodiment, it is possible to realize a light detection system capable of acquiring images of higher quality.
[0099] [Fourth embodiment] A range image sensor according to a fourth embodiment of the present invention will be described with reference to Fig. 14. Fig. 14 is a block diagram showing a schematic configuration of the range image sensor according to this embodiment. In this embodiment, the range image sensor will be described as an example of a light detection system to which the photoelectric conversion device 120 described in the first or second embodiment is applied.
[0100] 14, the range image sensor 300 according to this embodiment may include an optical system 302, a photoelectric conversion device 304, an image processing circuit 306, a monitor 308, and a memory 310. This range image sensor 300 receives light (modulated light or pulsed light) that is irradiated from a light source device 320 toward a subject 330 and reflected by the surface of the subject 330, and obtains a range image according to the distance to the subject 330.
[0101] The optical system 302 is composed of one or more lenses, and serves to focus image light (incident light) from a subject 330 on the light receiving surface (sensor section) of the photoelectric conversion device 304 .
[0102] The photoelectric conversion device 304 is the photoelectric conversion device 120 described in either the first or second embodiment, and has the function of generating a distance signal indicating the distance to the subject 330 based on the image light from the subject 330, and supplying the generated distance signal to the image processing circuit 306.
[0103] The image processing circuit 306 has a function of performing image processing to construct a distance image based on the distance signal supplied from the photoelectric conversion device 304 .
[0104] The monitor 308 has a function of displaying a distance image (image data) obtained by image processing in the image processing circuit 306. In addition, the memory 310 has a function of storing (recording) the distance image (image data) obtained by image processing in the image processing circuit 306.
[0105] Thus, according to this embodiment, by constructing a distance image sensor using the photoelectric conversion device of the first or second embodiment, it is possible to realize a distance image sensor that, in combination with improved characteristics of pixel 12, can acquire a distance image including more accurate distance information.
[0106] [Fifth embodiment] An endoscopic surgery system according to a fifth embodiment of the present invention will be described with reference to Fig. 15. Fig. 15 is a schematic diagram showing a configuration example of the endoscopic surgery system according to this embodiment. In this embodiment, the endoscopic surgery system will be described as an example of a light detection system to which the photoelectric conversion device 120 described in the first or second embodiment is applied.
[0107] FIG. 15 shows a state in which an operator (doctor) 460 is performing surgery on a patient 472 lying on a patient bed 470 using an endoscopic surgery system 400.
[0108] The endoscopic surgery system 400 of this embodiment may include an endoscope 410, a surgical tool 420, and a cart 430 on which various devices for endoscopic surgery are mounted, as shown in Fig. 15. The cart 430 may be mounted with a CCU (camera control unit) 432, a light source device 434, an input device 436, a treatment tool control device 438, a display device 440, and the like.
[0109] The endoscope 410 includes a lens barrel 412, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 472, and a camera head 414 connected to the base end of the lens barrel 412. Although Fig. 15 illustrates the endoscope 410 configured as a so-called rigid lens barrel having a rigid lens barrel 412, the endoscope 410 may be configured as a so-called flexible lens barrel having a flexible lens barrel. The endoscope 410 is held in a movable state by an arm 416.
[0110] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 412. A light source device 434 is connected to the endoscope 410, and light generated by the light source device 434 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 412, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 472. The endoscope 410 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
[0111] An optical system and a photoelectric conversion device (not shown) are provided inside the camera head 414, and reflected light (observation light) from an observation target is collected by the optical system onto the photoelectric conversion device. The photoelectric conversion device photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The photoelectric conversion device can be the photoelectric conversion device 120 described in the first or second embodiment. The image signal is transmitted to the CCU 432 as RAW data.
[0112] The CCU 432 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 410 and the display device 440. Furthermore, the CCU 432 receives an image signal from the camera head 414, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.
[0113] Under the control of the CCU 432 , the display device 440 displays an image based on the image signal that has been subjected to image processing by the CCU 432 .
[0114] The light source device 434 is composed of a light source such as an LED (Light Emitting Diode), and supplies the endoscope 410 with irradiation light when photographing an operation site or the like.
[0115] The input device 436 is an input interface for the endoscopic surgery system 400. A user can input various information and instructions to the endoscopic surgery system 400 via the input device 436.
[0116] The treatment tool control device 438 controls the driving of an energy treatment tool 450 for cauterizing tissue, incising, sealing blood vessels, or the like.
[0117] The light source device 434 that supplies irradiation light to the endoscope 410 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 434. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation target with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 414 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.
[0118] Furthermore, the light source device 434 may be controlled to change the intensity of the light it outputs at predetermined time intervals. By controlling the driving of the image sensor of the camera head 414 in synchronization with the timing of the change in the light intensity to obtain images in a time-division manner and synthesizing the images, it is possible to generate an image with a high dynamic range that is free of so-called blackout and whiteout.
[0119] The light source device 434 may be configured to supply light in a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, the wavelength dependency of light absorption in body tissue is utilized. Specifically, a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast by irradiating light in a narrower band than the irradiation light (i.e., white light) in normal observation. Alternatively, the special light observation may be a fluorescent observation in which an image is obtained by fluorescence generated by irradiating excitation light. In the fluorescent observation, it is possible to irradiate excitation light to the body tissue and observe the fluorescence from the body tissue, or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 434 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.
[0120] Thus, according to this embodiment, by configuring an endoscopic surgery system using the photoelectric conversion device of the first or second embodiment, it is possible to realize an endoscopic surgery system capable of acquiring higher quality images.
[0121] [Sixth embodiment] A light detection system and a moving body according to a sixth embodiment of the present invention will be described with reference to Figs. 16 to 18. Fig. 16 is a schematic diagram showing a configuration example of a moving body according to this embodiment. Fig. 17 is a block diagram showing a schematic configuration of a light detection system according to this embodiment. Fig. 18 is a flow diagram showing the operation of the light detection system according to this embodiment. In this embodiment, an application example of a light detection system to which the photoelectric conversion device 120 described in the first or second embodiment is applied, to an in-vehicle camera, is shown.
[0122] FIG. 16 is a schematic diagram showing a configuration example of a moving body (vehicle system) according to this embodiment. FIG. 16 shows the configuration of a vehicle 500 (automobile) as an example of a vehicle system incorporating a light detection system to which the photoelectric conversion device 120 according to the first or second embodiment is applied. FIG. 16(a) is a schematic front view of the vehicle 500, FIG. 16(b) is a schematic plan view of the vehicle 500, and FIG. 16(c) is a schematic rear view of the vehicle 500. The vehicle 500 is provided with a pair of photoelectric conversion devices 502 on the front side. Here, the photoelectric conversion devices 502 are the photoelectric conversion devices 120 described in the first or second embodiment. The vehicle 500 also includes an integrated circuit 503, an alarm device 512, and a main control unit 513.
[0123] FIG. 17 is a block diagram showing a configuration example of a light detection system 501 mounted on a vehicle 500. The light detection system 501 includes a photoelectric conversion device 502, an image preprocessing unit 515, an integrated circuit 503, and an optical system 514. The photoelectric conversion device 502 is the photoelectric conversion device 120 described in the first or second embodiment. The optical system 514 forms an optical image of a subject on the photoelectric conversion device 502. The photoelectric conversion device 502 converts the optical image of the subject formed by the optical system 514 into an electrical signal. The image preprocessing unit 515 performs a predetermined signal processing on the signal output from the photoelectric conversion device 502. The function of the image preprocessing unit 515 may be incorporated in the photoelectric conversion device 502. The light detection system 501 is provided with at least two sets of the optical system 514, the photoelectric conversion device 502, and the image preprocessing unit 515, and the output from each set of the image preprocessing unit 515 is input to the integrated circuit 503.
[0124] The integrated circuit 503 is an integrated circuit for use in an imaging system, and includes an image processing unit 504, an optical distance measuring unit 506, a parallax calculation unit 507, an object recognition unit 508, and an abnormality detection unit 509. The image processing unit 504 processes an image signal output from an image pre-processing unit 515. For example, the image processing unit 504 performs image processing such as development processing and defect correction on the output signal of the image pre-processing unit 515. The image processing unit 504 includes a memory 505 that temporarily stores the image signal. For example, the positions of known defective pixels in the photoelectric conversion device 502 can be stored in the memory 505.
[0125] The optical distance measuring unit 506 performs focusing and distance measurement of a subject. The parallax calculation unit 507 calculates distance measurement information (distance information) from a plurality of image data (parallax images) acquired by a plurality of photoelectric conversion devices 502. Each of the photoelectric conversion devices 502 may have a configuration capable of acquiring various information such as distance information. The object recognition unit 508 recognizes subjects such as cars, roads, signs, and people. When the abnormality detection unit 509 detects an abnormality in the photoelectric conversion device 502, it notifies the main control unit 513 of the abnormality.
[0126] The integrated circuit 503 may be realized by dedicated hardware, a software module, or a combination of these. It may also be realized by a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or a combination of these.
[0127] The main control unit 513 supervises and controls the operations of the light detection system 501, the vehicle sensor 510, the control unit 520, etc. The vehicle 500 does not necessarily have to include the main control unit 513. In this case, the photoelectric conversion device 502, the vehicle sensor 510, and the control unit 520 transmit and receive control signals via a communication network. For example, the CAN standard may be applied to the transmission and reception of these control signals.
[0128] The integrated circuit 503 has a function of receiving a control signal from the main control unit 513 or transmitting a control signal or a set value to the photoelectric conversion device 502 by its own control unit.
[0129] The optical detection system 501 is connected to a vehicle sensor 510, and can detect the vehicle's driving state, such as vehicle speed, yaw rate, and steering angle, as well as the state of the environment outside the vehicle and other vehicles and obstacles. The vehicle sensor 510 is also a distance information acquisition means for acquiring distance information to an object. The optical detection system 501 is also connected to a driving assistance control unit 511 that performs various driving assistance functions, such as automatic steering, automatic cruising, and collision prevention functions. In particular, the collision determination function determines whether or not a collision with another vehicle or obstacle has occurred based on the detection results of the optical detection system 501 and the vehicle sensor 510. This allows for avoidance control when a collision is estimated, and activation of a safety device in the event of a collision.
[0130] The light detection system 501 is also connected to an alarm device 512 that issues an alarm to the driver based on the result of the determination by the collision determination unit. For example, if the collision determination unit determines that there is a high possibility of a collision, the main control unit 513 performs vehicle control to avoid a collision and reduce damage by applying the brakes, releasing the accelerator, suppressing engine output, etc. The alarm device 512 warns the user by sounding an alarm, displaying alarm information on a display screen such as a car navigation system or a meter panel, vibrating a seat belt or steering wheel, etc.
[0131] In this embodiment, the surroundings of the vehicle, for example, the front or rear, are photographed by the light detection system 501. Fig. 16(b) shows an example of the arrangement of the light detection system 501 when the light detection system 501 photographs the area in front of the vehicle.
[0132] As described above, the photoelectric conversion device 502 is disposed in front of the vehicle 500. Specifically, if the center line of the vehicle 500 with respect to its forward / backward direction or outer shape (for example, vehicle width) is regarded as an axis of symmetry, and the two photoelectric conversion devices 502 are disposed in line symmetry with respect to the axis of symmetry, this is preferable for obtaining distance information between the vehicle 500 and an object to be photographed and for determining the possibility of a collision. In addition, the photoelectric conversion device 502 is preferably disposed so as not to obstruct the driver's field of vision when the driver visually checks the situation outside the vehicle 500 from the driver's seat. The warning device 512 is preferably disposed so as to be easily within the driver's field of vision.
[0133] Next, a fault detection operation of the photoelectric conversion device 502 in the light detection system 501 will be described with reference to Fig. 18. The fault detection operation of the photoelectric conversion device 502 can be performed according to steps S110 to S180 shown in Fig. 18.
[0134] Step S110 is a step for performing startup settings of the photoelectric conversion device 502. That is, settings for the operation of the photoelectric conversion device 502 are transmitted from outside the photodetection system 501 (e.g., the main control unit 513) or from inside the photodetection system 501, and the image capturing operation and fault detection operation of the photoelectric conversion device 502 are started.
[0135] Next, in step S120, pixel signals are obtained from the valid pixels. In addition, in step S130, output values are obtained from failure detection pixels provided for failure detection. The failure detection pixels have photoelectric conversion elements, just like the valid pixels. A predetermined voltage is written to the photoelectric conversion elements. The failure detection pixels output signals corresponding to the voltage written to the photoelectric conversion elements. Note that steps S120 and S130 may be reversed.
[0136] Next, in step S140, a judgment is made as to whether the output expected value of the fault detection pixel corresponds to the output value from the actual fault detection pixel. If the result of the judgment in step S140 indicates that the output expected value and the actual output value correspond to each other, the process proceeds to step S150, where it is judged that the imaging operation is performed normally, and the process proceeds to step S160. In step S160, the pixel signal of the scanning row is sent to the memory 505 and temporarily stored. Thereafter, the process returns to step S120, and the fault detection operation is continued. On the other hand, if the result of the judgment in step S140 indicates that the output expected value and the actual output value do not correspond to each other, the process proceeds to step S170. In step S170, it is judged that there is an abnormality in the imaging operation, and an alarm is notified to the main control unit 513 or the alarm device 512. The alarm device 512 displays the fact that an abnormality has been detected on the display unit. Thereafter, in step S180, the photoelectric conversion device 502 is stopped, and the operation of the light detection system 501 is terminated.
[0137] In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for each set of lines, or the fault detection operation may be performed for each frame. The issuance of the alarm in step S170 may be notified to the outside of the vehicle via a wireless network.
[0138] In addition, in this embodiment, the control to prevent collision with other vehicles has been described, but the control can also be applied to automatic driving control to follow other vehicles, automatic driving control not to go out of the lane, etc. Furthermore, the light detection system 501 is not limited to vehicles such as the vehicle itself, but can be applied to moving bodies (moving devices) such as ships, aircraft, and industrial robots. In addition, the light detection system 501 is not limited to moving bodies, but can be applied to a wide range of devices that use object recognition, such as intelligent transport systems (ITS).
[0139] [Seventh embodiment] A light detection system according to a seventh embodiment of the present invention will be described with reference to Fig. 19. Fig. 19 is a schematic diagram showing a configuration example of the light detection system according to this embodiment. In this embodiment, an application example to glasses (smart glasses) will be described as a light detection system to which the photoelectric conversion device 120 described in the first or second embodiment is applied.
[0140] 19(a) shows glasses 600 (smart glasses) according to one application example. The glasses 600 include a lens 601, a photoelectric conversion device 602, and a control device 603.
[0141] The photoelectric conversion device 602 is the photoelectric conversion device 120 described in the first or second embodiment, and is provided on the lens 601. The number of photoelectric conversion devices 602 may be one or more. When a plurality of photoelectric conversion devices 602 are used, a combination of a plurality of types of photoelectric conversion devices 602 may be used. The arrangement position of the photoelectric conversion device 602 is not limited to that shown in FIG. 19(a). A display device (not shown) including a light-emitting device such as an OLED or LED may be provided on the back side of the lens 601.
[0142] The control device 603 functions as a power source that supplies power to the photoelectric conversion device 602 and the display device. The control device 603 also has a function of controlling the operations of the photoelectric conversion device 602 and the display device. The lens 601 is provided with an optical system for collecting light on the photoelectric conversion device 602.
[0143] 19(b) shows glasses 610 (smart glasses) according to another application example. The glasses 610 include lenses 611 and a control device 612. The control device 612 may be equipped with a photoelectric conversion device (not shown) corresponding to the photoelectric conversion device 602 and a display device.
[0144] The lens 611 is provided with a photoelectric conversion device in the control device 612 and an optical system for projecting light from the display device, and an image is projected. The control device 612 functions as a power source for supplying power to the photoelectric conversion device and the display device, and also has a function for controlling the operations of the photoelectric conversion device and the display device.
[0145] The control device 612 may further include a line-of-sight detection unit that detects the line of sight of the wearer. In this case, an infrared light emitting unit may be provided in the control device 612, and infrared light emitted from the infrared light emitting unit may be used to detect the line of sight. Specifically, the infrared light emitting unit emits infrared light toward the eyeball of the user gazing at the display image. An imaging unit having a light receiving element detects the reflected light of the emitted infrared light from the eyeball, thereby obtaining an image of the eyeball. By providing a reduction means for reducing the light from the infrared light emitting unit to the display unit in a planar view, it is possible to reduce degradation of image quality.
[0146] The user's line of sight with respect to the displayed image can be detected from a captured image of the eyeball obtained by capturing infrared light. Any known method can be applied to gaze detection using a captured image of the eyeball. As an example, a gaze detection method based on a Purkinje image formed by reflection of irradiated light on the cornea can be used. More specifically, a gaze detection process based on a pupil-corneal reflex method is performed. Using the pupil-corneal reflex method, a gaze vector representing the direction (rotation angle) of the eyeball is calculated based on the image of the pupil and the Purkinje image included in the captured image of the eyeball, thereby detecting the user's gaze.
[0147] The display device of this embodiment may be configured to include a photoelectric conversion device having a light receiving element, and to control a display image based on the user's line of sight information from the photoelectric conversion device. Specifically, the display device determines a first field of view area to which the user gazes and a second field of view area other than the first field of view area based on the line of sight information. The first field of view area and the second field of view area may be determined by a control device of the display device, or may be determined by an external control device. When determined by an external control device, they are communicated to the display device via communication. In the display area of the display device, the display resolution of the first field of view area may be controlled to be higher than the display resolution of the second field of view area. In other words, the resolution of the second field of view area may be lower than the resolution of the first field of view area.
[0148] The display area may also be configured to have a first display area and a second display area different from the first display area, and to determine an area having a high priority from the first display area and the second display area based on line-of-sight information. The first display area and the second display area may be determined by a control device of the display device, or may be determined by an external control device. When determined by an external control device, the determination is communicated to the display device via communication. The resolution of the area having a high priority may be controlled to be higher than the resolution of areas other than the area having a high priority. In other words, the resolution of an area having a relatively low priority may be lowered.
[0149] AI may be used to determine the first field of view area and areas with high priority. The AI may be a model configured to estimate the angle of the line of sight and the distance to an object at the end of the line of sight from the image of the eyeball, using as teacher data an image of the eyeball and the direction in which the eyeball in the image was actually looking. The AI program may be included in the display device, the photoelectric conversion device, or an external device. If included in the external device, it is transmitted to the display device via communication.
[0150] When display control is performed based on visual recognition detection, the present invention is preferably applicable to smart glasses further including a photoelectric conversion device for capturing an image of the outside world. The smart glasses can display captured outside information in real time.
[0151] [Modified embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, an example in which part of the configuration of any one of the embodiments is added to another embodiment, or an example in which part of the configuration of another embodiment is substituted therefor, is also an embodiment of the present invention.
[0152] In addition, in the above embodiment, the count values were obtained from the counting unit 32 at times T0, T1, and T2, but the number of times that the count values are obtained from the counting unit 32 during one accumulation period is not limited to three times, and may be two times, or four or more times.
[0153] In the above embodiment, a recharge method is applied in which the photoelectric conversion element PD is periodically recharged as the photoelectric conversion unit 30, but the photoelectric conversion unit 30 does not necessarily have to adopt the recharge method. For example, the switch unit MP driven by a periodic signal may be replaced with a resistive element or an active quench circuit.
[0154] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0155] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.
[0156] The disclosure of the above embodiment includes the following configurations. (Configuration 1) a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counter that counts the photon detection signal output from the photoelectric conversion unit; A counting control unit that controls a counting operation in the counting unit, The count control unit resets the count value and restarts the count operation from an initial value when a count value from the start of the count operation to a first time is less than a first threshold value and a count value from the start of the count operation to a second time that is later than the first time is equal to or greater than a second threshold value that is greater than the first threshold value. A photoelectric conversion device comprising: (Configuration 2) The count control unit stops the counting operation when the count value up to the first time is equal to or greater than the first threshold, when the count value up to the second time is equal to or greater than the first threshold, or when the count value up to the first time is less than the first threshold and the count value up to the second time is equal to or greater than the first threshold and less than the second threshold. 2. The photoelectric conversion device according to configuration 1. (Configuration 3) The count control unit continues the counting operation when the count value up to the second time is less than the first threshold value. 3. The photoelectric conversion device according to configuration 1 or 2. (Configuration 4) When a count value from the start of the counting operation to a third time that is later than the second time is equal to or greater than the second threshold value, the counting control unit resets the count value and restarts the counting operation from an initial value. 4. The photoelectric conversion device according to configuration 3. (Configuration 5) The count control unit stops the counting operation when the count value from the start of the counting operation to a third time that is later than the second time is equal to or greater than the first threshold value and less than the second threshold value. 4. The photoelectric conversion device according to configuration 3. (Configuration 6) The count control unit stops the counting operation when the count value reaches an upper count limit value of the counting unit. 6. The photoelectric conversion device according to any one of configurations 1 to 5. (Configuration 7) a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counter that counts the photon detection signal output from the photoelectric conversion unit; A counting control unit that controls a counting operation in the counting unit, The counting control unit is a first memory that stores a comparison result between a count value from the start of the counting operation to a first time and a first threshold value; a second memory that stores a comparison result between the count value from the start of the counting operation to a second time that is later than the first time and the first threshold value; a third memory that stores a comparison result between the count value from the start of the count operation to the second time and a second threshold value that is greater than the first threshold value; The counting operation is controlled based on the information stored in the first memory, the second memory, and the third memory. A photoelectric conversion device comprising: (Configuration 8) The count control unit executes any one of continuing the counting operation, stopping the counting operation, and restarting the counting operation based on the information stored in the first memory, the second memory, and the third memory. 8. The photoelectric conversion device according to configuration 7. (Configuration 9) The count control unit resets the count value and restarts the counting operation from an initial value when the first memory indicates that the count value up to the first time is less than the first threshold, the second memory indicates that the count value up to the second time is equal to or greater than the first threshold, and the third memory indicates that the count value up to the second time is equal to or greater than the second threshold. 9. The photoelectric conversion device according to configuration 7 or 8. (Configuration 10) The count control unit continues the counting operation when the first memory indicates that the count value up to the first time is less than the first threshold value and the second memory indicates that the count value up to the second time is less than the first threshold value. 10. The photoelectric conversion device according to any one of configurations 7 to 9. (Configuration 11) The counting control unit is a fourth memory that stores a comparison result between the count value from the start of the counting operation to a third time that is later than the second time and the first threshold value; a fifth memory that stores a comparison result between the count value from the start of the count operation to the third time and the second threshold value, When the fourth memory indicates that the count value up to the third time is equal to or greater than the first threshold value and the fifth memory indicates that the count value up to the third time is equal to or greater than the second threshold value, the count value is reset and the counting operation is restarted from an initial value. 11. The photoelectric conversion device according to configuration 10. (Configuration 12) The count control unit stops the counting operation when the first memory indicates that the count value up to the first time is equal to or greater than the first threshold value, or when the second memory indicates that the count value up to the second time is equal to or greater than the first threshold value. 10. The photoelectric conversion device according to any one of configurations 7 to 9. (Configuration 13) The count control unit stops the counting operation when the count value reaches an upper count limit value of the counting unit, regardless of the information stored in the first memory, the second memory, and the third memory. 13. The photoelectric conversion device according to any one of configurations 7 to 12. (Configuration 14) the photoelectric conversion unit includes an avalanche photodiode and a control circuit that periodically controls the avalanche photodiode in response to a periodic signal between a standby state in which avalanche multiplication is possible and a recharge state in which the avalanche photodiode is returned to a state in which avalanche multiplication is possible again after avalanche multiplication has occurred; The counter counts the photon detection signal outputted corresponding to a period during which the avalanche multiplication occurs within a period during which the avalanche photodiode is in the standby state. 14. The photoelectric conversion device according to any one of configurations 1 to 13. (Configuration 15) a plurality of pixels each having the photoelectric conversion unit, the counting unit, and the count control unit; 15. The photoelectric conversion device according to any one of configurations 1 to 14. (Configuration 16) A photoelectric conversion device according to any one of structures 1 to 15, a signal processing device that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising: (Configuration 17) The signal processing device generates a distance image representing distance information to an object based on the signal. 17. The optical detection system of claim 16. (Configuration 18) A mobile object, A photoelectric conversion device according to any one of structures 1 to 15, a distance information acquisition unit that acquires distance information to an object from a parallax image based on a signal output from the photoelectric conversion device; a control means for controlling the moving object based on the distance information; A moving object comprising: [Explanation of symbols]
[0157] PD: Photoelectric conversion element LC1, LC2, LC3...Latch circuits MEM0, MEM1, MEM2, MEM3, MEM4...Memory 10…Pixel area 12…pixels 30...Photoelectric conversion section 32…Counting section 34...Counting control section 341,342…Comparison section 343,344...Storage section 345…Recount judgment section 346…Stop signal generating unit 347...Reset signal generator 120...Photoelectric conversion device
Claims
1. a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counter that counts the photon detection signal output from the photoelectric conversion unit; A counting control unit that controls a counting operation in the counting unit, The count control unit resets the count value and restarts the count operation from an initial value when a count value from the start of the count operation to a first time is less than a first threshold value and a count value from the start of the count operation to a second time that is later than the first time is equal to or greater than a second threshold value that is greater than the first threshold value. A photoelectric conversion device comprising:
2. The count control unit stops the counting operation when the count value up to the first time is equal to or greater than the first threshold, when the count value up to the second time is equal to or greater than the first threshold, or when the count value up to the first time is less than the first threshold and the count value up to the second time is equal to or greater than the first threshold and less than the second threshold.
2. The photoelectric conversion device according to claim 1.
3. The count control unit continues the counting operation when the count value up to the second time is less than the first threshold value.
3. The photoelectric conversion device according to claim 1 or 2.
4. When a count value from the start of the counting operation to a third time that is later than the second time is equal to or greater than the second threshold value, the counting control unit resets the count value and restarts the counting operation from an initial value.
4. The photoelectric conversion device according to claim 3.
5. The count control unit stops the counting operation when the count value from the start of the counting operation to a third time that is later than the second time is equal to or greater than the first threshold value and less than the second threshold value.
4. The photoelectric conversion device according to claim 3.
6. The count control unit stops the counting operation when the count value reaches an upper count limit value of the counting unit.
3. The photoelectric conversion device according to claim 1 or 2.
7. a photoelectric conversion unit that outputs a photon detection signal in response to incidence of a photon; a counter that counts the photon detection signal output from the photoelectric conversion unit; A counting control unit that controls a counting operation in the counting unit, The counting control unit is a first memory that stores a result of comparing a count value from the start of the counting operation to a first time point with a first threshold value; a second memory that stores a result of a comparison between a count value from a start of the counting operation to a second time that is later than the first time and the first threshold value; a third memory that stores a result of a comparison between the count value from the start of the counting operation to the second time and a second threshold value that is greater than the first threshold value; The counting operation is controlled based on the information stored in the first memory, the second memory, and the third memory. A photoelectric conversion device comprising:
8. The count control unit executes any one of continuing the counting operation, stopping the counting operation, and restarting the counting operation based on the information stored in the first memory, the second memory, and the third memory.
8. The photoelectric conversion device according to claim 7.
9. The count control unit resets the count value and restarts the counting operation from an initial value when the first memory indicates that the count value up to the first time is less than the first threshold value, the second memory indicates that the count value up to the second time is equal to or greater than the first threshold value, and the third memory indicates that the count value up to the second time is equal to or greater than the second threshold value.
9. The photoelectric conversion device according to claim 7 or 8.
10. The count control unit continues the counting operation when the first memory indicates that the count value up to the first time is less than the first threshold value and the second memory indicates that the count value up to the second time is less than the first threshold value.
9. The photoelectric conversion device according to claim 7 or 8.
11. The counting control unit is a fourth memory that stores a result of a comparison between a count value from a start of the counting operation to a third time that is later than the second time and the first threshold value; a fifth memory that stores a result of a comparison between the count value from the start of the counting operation to the third time and the second threshold value, When the fourth memory indicates that the count value up to the third time is equal to or greater than the first threshold value and the fifth memory indicates that the count value up to the third time is equal to or greater than the second threshold value, the count value is reset and the counting operation is restarted from an initial value.
11. The photoelectric conversion device according to claim 10.
12. The count control unit stops the counting operation when the first memory indicates that the count value up to the first time is equal to or greater than the first threshold value, or when the second memory indicates that the count value up to the second time is equal to or greater than the first threshold value.
9. The photoelectric conversion device according to claim 7 or 8.
13. The count control unit stops the counting operation when the count value reaches an upper count limit value of the counting unit, regardless of the information stored in the first memory, the second memory, and the third memory.
9. The photoelectric conversion device according to claim 7 or 8.
14. the photoelectric conversion unit includes an avalanche photodiode and a control circuit that periodically controls the avalanche photodiode in response to a periodic signal between a standby state in which avalanche multiplication is possible and a recharge state in which the avalanche photodiode is returned to a state in which avalanche multiplication is possible again after avalanche multiplication has occurred; The counter counts the photon detection signal outputted corresponding to a period during which the avalanche multiplication occurs within a period during which the avalanche photodiode is in the standby state.
9. The photoelectric conversion device according to claim 1, 2, 7, or 8.
15. a plurality of pixels each having the photoelectric conversion unit, the counting unit, and the count control unit; 9. The photoelectric conversion device according to claim 1, 2, 7, or 8.
16. The photoelectric conversion device according to claim 1 , 2 , 7 , or 8 , a signal processing device that processes a signal output from the photoelectric conversion device; 1. An optical detection system comprising:
17. The signal processing device generates a distance image representing distance information to an object based on the signal.
17. The optical detection system of claim 16.
18. A mobile object, The photoelectric conversion device according to claim 1 , 2 , 7 , or 8 , a distance information acquisition unit that acquires distance information to an object from a parallax image based on a signal output from the photoelectric conversion device; a control means for controlling the moving object based on the distance information; A moving object comprising:
Citation Information
Patent Citations
Photoelectric conversion device and photoelectric conversion system
JP2023039400A
