Photoelectric conversion device and photoelectric conversion system

Abstract

To provide a photoelectric conversion device that enables high-resolution event detection while reducing circuit size.SOLUTION: A photoelectric conversion device has a pixel unit having a photoelectric conversion unit that outputs a pulse signal in response to incidence of light, a count unit having a counter that counts the pulse signals output from the photoelectric conversion unit during a predetermined exposure period, a count control unit that controls the count unit, and a determination unit that performs a determination process to determine whether there is a change in the amount of light incident on the photoelectric conversion unit on the basis of the count value of the counter. The exposure period has a plurality of periods including at least a first period and a second period, the count control unit controls the count unit to output a difference value between a first count value in the first period and a second count value in the second period, and the determination unit performs the determination process on the basis of the difference value.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a photoelectric conversion device and a photoelectric conversion system.

Background Art

[0002] With the spread of technologies such as IoT, AI, and autonomous driving in recent years, there has been a demand for image sensors that consume less power and can operate at high speed compared to the conventional ones. One such example is an event-based sensor (also called a dynamic vision sensor). In this sensor, each of the event detection pixels arranged in a two-dimensional array monitors the change in the incident light amount. When the incident light amount increases (occurrence of an on-event), decreases (occurrence of an off-event), or does not change (no event occurs), information on any of these cases is generated. By configuring to output this information only from the event pixels where an on-event or an off-event has occurred, it is possible to achieve low power consumption and high-speed operation.

[0003] On the other hand, since the output of this event detection pixel is three-valued data, the image quality deteriorates as compared with the conventional synchronous solid-state imaging device in terms of image data. From such a viewpoint, Patent Document 1 discloses a technique of combining an event detection pixel and a counting pixel capable of outputting multi-bit luminance information to generate higher-quality image data while detecting events. In the solid-state imaging device described in Patent Document 1, by using a SPAD (Single Photon Avalanche Diode) pixel capable of counting the number of photons, the image quality at low illuminance is improved.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when attempting to configure an event detection pixel using an SPAD for the purpose of improving event detection sensitivity in low light conditions, etc., a memory for storing a count value, a subtractor, a comparator, etc. are required, resulting in an increase in the circuit scale of the event detection pixel. As a result, the resolution of the event detection pixel may decrease, or the resolution of the counting pixel may decrease by increasing the event detection pixel.

[0006] An object of the present invention is to provide a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

Means for Solving the Problems

[0007] According to one disclosure of this specification, a photoelectric conversion unit that outputs a pulse signal in response to the incidence of light, a counting unit having a counter that counts the pulse signal output from the photoelectric conversion unit during a predetermined exposure period, a count control unit that controls the counting unit, and a determination unit that performs a determination process for determining the presence or absence of a change in the amount of light incident on the photoelectric conversion unit based on the count value of the counter, and a pixel unit having the above components, the exposure period has a plurality of periods including at least a first period and a second period, the count control unit controls the counting unit to output a difference value between a first count value in the first period and a second count value in the second period, and the determination unit performs the determination process based on the difference value, and a photoelectric conversion device is provided.

Effects of the Invention

[0008] According to the present invention, it is possible to realize a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Embodiments for Carrying Out the Invention

[0010] [First Embodiment] The photoelectric conversion device according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 5. FIG. 1 is a block diagram showing a schematic configuration of the photoelectric conversion device according to the present embodiment. FIG. 2 is a diagram showing an example of a color filter array of the photoelectric conversion device according to the present embodiment. FIG. 3 is a block diagram showing a configuration example of a pixel portion of the photoelectric conversion device according to the present embodiment. FIG. 4 is a circuit diagram showing a configuration example of a photoelectric conversion portion of the photoelectric conversion device according to the present embodiment. FIG. 5 is a block diagram showing a configuration example of a count unit, a count control unit, and a determination unit of the photoelectric conversion device according to the present embodiment.

[0011] The photoelectric conversion device according to this embodiment can be configured by, for example, a pixel array unit 10, a readout unit 60, and a control unit 70 as shown in FIG. 1. The pixel array unit 10 is connected to the readout unit 60. The control unit 70 is connected to the pixel array unit 10 and the readout unit 60.

[0012] The pixel array unit 10 is provided with a plurality of pixel units 12 arranged in a plurality of rows and a plurality of columns. Each of the plurality of pixel units 12 has a function of detecting a change in the incident light amount as event information. Note that the number of pixel units 12 constituting the pixel array unit 10 is not particularly limited. For example, the pixel array unit 10 can be configured by a plurality of pixel units 12 arranged in an array of several thousand rows × several thousand columns like a general digital camera. Alternatively, the pixel array unit 10 may be configured by a plurality of pixel units 12 arranged in one row or one column. Alternatively, the pixel array unit 10 may be configured by one pixel unit 12.

[0013] The pixel array unit 10 may be provided with color filters (hereinafter referred to as CF) having various arrays as shown in FIG. 2, for example. FIG. 2 shows an example of the CF array in a 4-column × 4-row pixel unit 12. FIG. 2(a) shows a case where all pixel units 12 are composed of clear (C) pixels not provided with CF. According to the configuration of FIG. 2(a), the sensitivity in a dark place can be increased. FIG. 2(b) shows a CF array called a Bayer array, which is a case of an RGGB array including an R pixel having a CF that transmits light in the red (R) wavelength range, a G pixel having a CF that transmits light in the green (G) wavelength range, and a B pixel having a CF that transmits light in the blue (B) wavelength range. According to the configuration of FIG. 2(b), it is possible to acquire information for each color. FIG. 2(c) shows a case of an RGCB array in which a part of the G pixels in the Bayer array is replaced with C pixels. FIG. 2(d) shows a case of an RCCB array in which the G pixels in the Bayer array are replaced with C pixels. According to the configurations of FIGS. 2(c) and 2(d), in addition to information for each color, it is possible to acquire high-sensitivity information by the C pixels. Note that the CF array can be appropriately selected according to the application in which the photoelectric conversion device is used, etc., and is not limited to those exemplified in FIG. 2.

[0014] The readout unit 60 has a function of reading out the event information detected by the pixel array unit 10 and outputting it to an external device (not shown). The information may be read out pixel unit 12 by pixel unit 12, or may be read out in parallel and simultaneously from a plurality of pixel units 12. When reading out simultaneously from a plurality of pixel units 12, for example, it is possible to read out in units of rows or columns. Alternatively, for example, a configuration may be adopted in which only the row including the pixel unit 12 that has detected an event is read out. In this case, in order to specify the position of the pixel unit 12 that has detected an event, the row address may be added to the event value and output. Similarly, when only the column in which an event has been detected is to be read out, the column address may be added to the event value and output. Furthermore, a configuration may be adopted in which rows and columns are combined and read out for each pixel block (a rectangular area including a plurality of pixel units 12) in which an event has been detected.

[0015] The control unit 70 has a function of controlling the operations and their timings of the pixel array unit 10 and the readout unit 60. The control unit 70 generates control signals for controlling the operations and their timings of the pixel array unit 10 and the readout unit 60, and outputs the generated control signals to the pixel array unit 10 and the readout unit 60. The control signals supplied from the control unit 70 to the pixel array unit 10 and the readout unit 60 may include, for example, a reset signal for initializing the pixel unit 12, a synchronization signal for determining the exposure period, a timing signal for controlling the readout period, and setting signals such as a threshold value for detecting an event.

[0016] Next, a configuration example of the pixel unit 12 constituting the pixel array unit 10 will be described with reference to FIG. 3. Each of the pixel units 12 may be constituted by, for example, a photoelectric conversion unit 20, a counting unit 30, a count control unit 40, and a determination unit 50 as shown in FIG. 3. Control signals from the control unit 70 are supplied to the photoelectric conversion unit 20 and the count control unit 40. The photoelectric conversion unit 20 and the count control unit 40 are connected to the counting unit 30. The counting unit 30 is connected to the determination unit 50. The determination unit 50 is connected to the count control unit 40 and the readout unit 60.

[0017] The control signal supplied from the control unit 70 to the photoelectric conversion unit 20 may include a signal CLKB that is an inverted signal of the synchronization signal CLK. The signal CLKB can be used as a synchronization signal for determining the exposure period of the photoelectric conversion unit 20. The control signal supplied from the control unit 70 to the count control unit 40 may include the signal CLKB and a reset signal RES of the counter.

[0018] The photoelectric conversion unit 20 outputs to the counting unit 30 a number of pulse signals corresponding to the amount of incident light during the exposure period defined by the signal CLKB. The counting unit 30 counts the pulse signals output from the photoelectric conversion unit 20 according to the control signal from the count control unit 40. The counting unit 30 performs at least once an exposure period for increasing (up-counting) the count value and an exposure period for decreasing (down-counting) the count value, and outputs the difference information of the count values in these exposure periods as the count value of the pixel unit 12. The count control unit 40 generates a control signal for controlling the count value and the count direction of the counting unit 30 according to the signal CLKB and the reset signal RES supplied from the control unit 70, and outputs it to the counting unit 30. The determination unit 50 determines the presence or absence of a change in the amount of light based on the count value output from the counting unit 30, and outputs the determination result as event information.

[0019] Next, a configuration example of the photoelectric conversion unit 20 constituting the pixel unit 12 will be described with reference to FIG. 4. The photoelectric conversion unit 20 may be composed of, for example, a photoelectric conversion element 22, a switch unit 24, and a waveform shaping unit 26 as shown in FIG. 4. The photoelectric conversion element 22 may be composed of an avalanche photodiode (hereinafter referred to as "APD"). The switch unit 24 may be composed of a switch element such as a MOS transistor. In this embodiment, the case where the switch unit 24 is composed of an N-type MOS transistor will be described as an example, but the switch unit 24 can also be composed of a P-type MOS transistor. The waveform shaping unit 26 may be composed of a logic circuit such as an inverter circuit.

[0020] The anode of the APD that constitutes the photoelectric conversion element 22 is connected to the ground voltage node. The cathode of the APD that constitutes the photoelectric conversion element 22 is connected to the source of the N-type MOS transistor that constitutes the switch unit 24. The drain of the N-type MOS transistor that constitutes the switch unit 24 is connected to the node to which the voltage Vbias is supplied. The input node of the waveform shaping unit 26 is connected to the connection node (node A) between the photoelectric conversion element 22 and the switch unit 24. The output node (node B) of the waveform shaping unit 26 becomes the output node of the photoelectric conversion unit 20. A signal CLKB is input from the control unit 70 to the gate of the N-type MOS transistor that constitutes the switch unit 24.

[0021] The photoelectric conversion element 22 can be constituted by an APD as described above. The voltage Vbias is set so that a reverse bias voltage sufficient for the APD to perform an avalanche multiplication operation is applied to the APD. In one example, as the voltage Vbias, a positive high voltage of about +20V, for example, is applied. By supplying a reverse bias voltage sufficient for the avalanche multiplication operation to the APD, the charge generated by the photoelectric conversion due to the light incident on the APD causes avalanche multiplication, and an avalanche current is generated. The operating modes in the state where a reverse bias voltage is supplied to the APD include a Geiger mode and a linear mode. The Geiger mode is an operating mode in which the voltage applied between the anode and the cathode is a reverse bias voltage greater than the breakdown voltage of the APD. The linear mode is an operating mode in which the voltage applied between the anode and the cathode is a reverse bias voltage near or below the breakdown voltage of the APD. An APD operated in the Geiger mode is called a SPAD (Single Photon Avalanche Diode). The APD that constitutes the photoelectric conversion element 22 may be operated in the linear mode or in the Geiger mode.

[0022] Note that the voltage Vbias applied to the cathode side of the photoelectric conversion element 22 does not necessarily have to be a positive high voltage. For example, a power supply voltage of about 1V to several volts may be applied to the cathode side of the photoelectric conversion element 22, and a negative high voltage may be applied to the anode side of the photoelectric conversion element 22. Further, the reverse bias voltage applied between the anode and cathode of the photoelectric conversion element 22 does not necessarily have to be 20V, and can be appropriately set according to the breakdown voltage of the APD.

[0023] The switch unit 24 has a role of controlling the supply of the voltage Vbias to the APD. That is, the switch unit 24 supplies the voltage Vbias to the node A by turning on upon receiving a high-level signal CLKB from the control unit 70, putting the APD into a state where avalanche multiplication operation is possible. Further, the switch unit 24 stops the supply of the voltage Vbias to the node A by turning off upon receiving a low-level signal CLKB from the control unit 70, prohibiting the avalanche multiplication operation in the APD. Further, the switch unit 24 also has a role as a quenching element that utilizes the resistance component of the N-type MOS transistor, that is, a resistance element for stopping the avalanche multiplication phenomenon of the APD.

[0024] The waveform shaping unit 26 has a role of shaping the voltage waveform of the node A and converting it into a pulse signal (photon detection signal) indicating that photons have entered the photoelectric conversion element 22. That is, the waveform shaping unit 26 outputs a high-level signal (photon detection signal) when the voltage of the node A is lower than a predetermined determination threshold value, and outputs a low-level signal when the voltage of the node A is higher than the predetermined determination threshold value.

[0025] The operation of the photoelectric conversion unit 20 when a single photon is incident is described below. When the signal CLKB becomes high level and the switch unit 24 is turned on, a voltage Vbias is applied to the anode of the APD via the switch unit 24, and the node A becomes high level. When a photon is incident on the APD in this state, electron-hole pairs are generated by photoelectric conversion, and an avalanche multiplication phenomenon occurs using these charges as seeds. When an avalanche current flows through the APD due to the avalanche multiplication phenomenon, a voltage drop occurs due to the resistance component (quenching resistance) of the switch unit 24, and the bias voltage applied to the APD begins to drop. When the bias voltage drops to the breakdown voltage, the avalanche multiplication phenomenon stops, and the node A becomes low level. As a result, the avalanche current no longer flows through the APD, and the voltage Vbias is applied to the cathode of the APD again, and the node A returns to high level.

[0026] As a result, the voltage of the node A transitions from high level to low level with the incidence of photons on the APD, and then transitions from low level to high level. Thereby, the output signal of the waveform shaping unit 26, that is, the voltage of the node B, transitions from low level to high level in response to the change in the signal level of the node A, and then transitions from high level to low level. The pulse signal output in this way is a photon detection signal indicating that photons have been incident on the photoelectric conversion element 22.

[0027] Next, a configuration example of the count unit 30, the count control unit 40, and the determination unit 50 that constitute the pixel unit 12 will be described with reference to FIG. 5. The count unit 30 has, for example, as shown in FIG. 5, an up-down (UD) counter 32 capable of performing both up counting and down counting, and a memory 34 as a count value holding unit. Further, the determination unit 50 has a comparator 52.

[0028] The count control unit 40 receives the signal CLKB and the reset signal RES from the control unit 70, and the event value which is the output signal of the determination unit 50, and generates control signals CNT_RES, CNT_UD, CNT_WR, and CNT_RD. Here, the control signal CNT_RES is a reset signal for resetting the UD counter 32. The control signal CNT_UD is a signal for switching between the up - count and down - count of the UD counter 32. For example, the UD counter 32 performs an up - count when the control signal CNT_UD is at a high level, and performs a down - count when the control signal CNT_UD is at a low level. The control signal CNT_WR is a signal for controlling the writing of the count value to the memory 34. The control signal CNT_RD is a control signal for controlling the reading of the count value (MEM_CNT) stored in the memory 34 to the UD counter 32.

[0029] The UD counter 32 receives the pulse signal from the photoelectric conversion unit 20, the control signals CNT_RES, CNT_UD, CNT_RD from the count control unit 40, and the memory value MEM_CNT from the memory 34. The memory 34 receives the control signal CNT_WR from the count control unit 40 and the count value which is the output of the UD counter 32. The comparator 52 of the determination unit 50 receives the count value which is the output of the UD counter 32, the positive threshold th_p, and the negative threshold th_m.

[0030] The event value which is the output signal of the determination unit 50 is generated based on the comparison result in the comparator 52. That is, the comparator 52 compares the count value which is the output signal from the count unit 30 with the positive threshold th_p and the negative threshold th_m, and detects an event when the count value exceeds the positive threshold th_p or is less than the negative threshold th_m. For example, the event value can be defined as in the following formula (1).

Equation

[0031] Event value 1 indicates that a change has occurred in the pixel unit 12 to become brighter, that is, the occurrence of an on-event. Event value -1 indicates that a change has occurred in the pixel unit 12 to become darker, that is, the occurrence of an off-event. Event value 0 indicates that no event has occurred. Equation (1) is an example of the expression of event values, and other expressions may be used, such as assigning different numerical values to each of the states of on-event, off-event, and no event, and treating on-events and off-events as one type of event value without distinction. The event value is updated in synchronization with the rising edge (positive edge) of the signal CLKB.

[0032] Next, the driving method of the photoelectric conversion device according to the present embodiment will be described with reference to FIGS. 6 and 7. FIG. 6 is a flowchart showing the driving method of the photoelectric conversion device according to the present embodiment. FIG. 7 is a timing chart showing the driving method of the photoelectric conversion device according to the present embodiment. FIG. 7 shows the count value of the UD counter 32, the value stored in the memory 34, the event value that is the output of the determination unit 50, and the waveforms of the signal CLKB, the reset signal RES, and the control signals CNT_RES, CNT_UD, CNT_WR, and CNT_RD.

[0033] First, the count value of the UD counter 32 is reset to 0 (step S101). At time t1s, when the reset signal RES is supplied from the control unit 70 to the count control unit 40, the count control unit 40 generates the same control signal CNT_RES as the reset signal RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger.

[0034] Also at time t1s, it is assumed that the signal CLKB transitions from the low level to the high level. The count control unit 40 controls the control signal CNT_UD to the low level in response to the first rising edge of the signal CLKB after receiving the reset signal RES. The UD counter 32 starts downcounting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the low-level control signal CNT_UD (step S102).

[0035] At a subsequent time t1e, when the signal CLKB transitions from a high level to a low level, the count control unit 40 controls the control signal CNT_UD from a low level to a high level. Then, in response to the rising edge of the control signal CNT_UD, the count control unit 40 outputs a control signal CNT_WR. The memory 34 stores the count value of the UD counter 32 as a memory value MEM_CNT using the control signal CNT_WR as a trigger (step S103).

[0036] Assume that at a subsequent time t2s, the signal CLKB transitions from a low level to a high level. The count control unit 40 outputs a control signal CNT_RD in response to the rising edge of the signal CLKB when the control signal CNT_UD is at a high level. The UD counter 32 reads the memory value MEM_CNT from the memory 34 using the control signal CNT_RD as a trigger, and updates its own count value with the read memory value MEM_CNT (step S104). Note that when the count value before the update is the same as the memory value MEM_CNT, as in the case of the time t2s or the time t6s in FIG. 7, this step may be omitted.

[0037] Also at the time t2s, the UD counter 32 starts an up - count of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the high - level control signal CNT_UD, starting from the updated count value (step S105). Note that after the count period during which a down - count is performed (the period from time t1s to time t1e (period d1)), the control signal CNT_UD remains at a high level until an event is detected, and the UD counter 32 is set for up - count. For example, a down - count is performed during the period d1, while an up - count is performed during the period from time t2s to time t2e (period d2). As a result, as the count value at the time t2e, a difference value between the count value in the period d2 and the count value in the period d1 can be obtained.

[0038] At the subsequent time t2e, when the signal CLKB transitions from a high level to a low level, the determination unit 50 performs event determination processing based on the count value received from the counting unit 30 and the threshold values th_p and th_m (step S106). The determination unit 50 outputs, as an event value, the determination result based on the comparison result between the count value and the threshold values th_p and th_m to the count control unit 40.

[0039] The count control unit 40 controls the UD counter 32 according to the event value received from the determination unit 50. Specifically, when no event is detected, that is, when the event value is 0 (''NO'' in step S107), it returns to step S104 and updates the count value of the UD counter 32 with the memory value MEM_CNT read from the memory 34. In the driving example of FIG. 7, the event value is 0 at time t2e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t3s. Further, the event value is also 0 at the subsequent time t3e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t4s. The operations in the period d3 from time t3s to time t3e and the period d4 from time t4s to time t4e are the same as those in the period d2 from time t2s to time t2e.

[0040] When an event is detected, that is, when the event value is 1 or -1 (''YES'' in step S107), it proceeds to step S108. The count control unit 40 generates a control signal CNT_RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger. In the driving example of FIG. 7, since the count value is below the threshold value th_m at time t4e, the event value is -1, and the count value of the UD counter 32 is reset to 0 at the subsequent time t5s. After the reset of the UD counter 32, it returns to step S102. That is, after time t5s, the same processing as that from time t1s is performed.

[0041] At a subsequent time t5e, when signal CLKB transitions from a high level to a low level, count control unit 40 controls control signal CNT_UD from a low level to a high level. Then, in response to the rising edge of control signal CNT_UD, count control unit 40 generates control signal CNT_WR. Memory 34 stores the count value of UD counter 32 as memory value MEM_CNT using control signal CNT_WR as a trigger (step S103).

[0042] Assume that at a subsequent time t6s, signal CLKB transitions from a low level to a high level. Count control unit 40 generates control signal CNT_RD in response to the rising edge of signal CLKB when control signal CNT_UD is at a high level. UD counter 32 reads memory value MEM_CNT from memory 34 using control signal CNT_RD as a trigger, and updates its own count value with the read memory value MEM_CNT (step S104).

[0043] Also at time t6s, UD counter 32 starts an up - count of the pulse signal (photon detection signal) output from photoelectric conversion unit 20 in response to the high - level control signal CNT_UD, starting from the updated count value (step S105). Note that after the count period during which down - count is performed (the period from time t5s to time t5e (period d5)), control signal CNT_UD remains at a high level until an event is detected, and UD counter 32 is set for up - count. For example, down - count is performed during period d5, while up - count is performed during the period from time t6s to time t6e (period d6). As a result, as the count value at time t6e, the difference value between the count value in period d6 and the count value in period d5 can be obtained.

[0044] At the subsequent time t6e, when the signal CLKB transitions from a high level to a low level, the determination unit 50 performs event determination processing based on the count value received from the counting unit 30 and the threshold values th_p and th_m (step S106). The determination unit 50 outputs the determination result based on the comparison result between the count value and the threshold values th_p and th_m as an event value to the count control unit 40.

[0045] The count control unit 40 controls the UD counter 32 according to the event value received from the determination unit 50. Specifically, when no event is detected, that is, when the event value is 0 (''NO'' in step S107), it returns to step S104 and updates the count value of the UD counter 32 with the memory value MEM_CNT read from the memory 34. In the driving example of FIG. 7, the event value is 0 at time t6e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t7s. Furthermore, the event value is also 0 at the subsequent time t7e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t8s. The operations during the period d7 from time t7s to time t7e and the period d8 from time t8s to time t8e are the same as those during the period d2 from time t2s to time t2e.

[0046] When an event is detected, that is, when the event value is 1 or -1 (''YES'' in step S208), it proceeds to step S209. The count control unit 40 generates a control signal CNT_RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger. In the driving example of FIG. 7, since the count value exceeds the threshold value th_p at time t8e, the event value is 1, and the count value of the UD counter 32 is reset to 0 at the subsequent time t9s. After the reset of the UD counter 32, it returns to step S102. That is, after time t9s, the same processing as that from time t1s is performed.

[0047] As described above, in this embodiment, by performing up-down control of the counter, the difference value between the two exposure periods can be obtained as a count value. Therefore, a subtracter for calculating the difference value can be reduced, and the configuration of the photoelectric conversion device can be simplified.

[0048] Therefore, according to this embodiment, a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale can be realized.

[0049] [Second Embodiment] The photoelectric conversion device and its driving method according to the second embodiment of the present invention will be described with reference to FIGS. 8 to 10. FIG. 8 is a block diagram showing a configuration example of a count unit, a count control unit, and a determination unit of the photoelectric conversion device according to this embodiment. FIG. 9 is a flowchart showing a driving method of the photoelectric conversion device according to this embodiment. FIG. 10 is a timing diagram showing a driving method of the photoelectric conversion device according to this embodiment. The same components as those of the photoelectric conversion device according to the first embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified.

[0050] The photoelectric conversion device according to this embodiment is the same as the photoelectric conversion device according to the first embodiment except that the configuration of the count unit 30 is different. In this embodiment, the description will be centered on the points different from the photoelectric conversion device according to the first embodiment, and the description of the same points as those of the photoelectric conversion device according to the first embodiment will be omitted as appropriate.

[0051] As shown in FIG. 8, the count unit 30 of the photoelectric conversion device according to this embodiment includes a counter 36 capable of performing at least one of up counting and down counting, and a sign inversion unit 38 that inverts the sign of the count value output from the counter 36.

[0052] The count control unit 40 receives the signal CLKB and the reset signal RES from the control unit 70 and the event value which is the output signal of the determination unit 50, and generates control signals CNT_RES, CNT_WR, and CNT_RD. The counter 36 receives the pulse signal from the photoelectric conversion unit 20, the control signals CNT_RES and CNT_RD from the count control unit 40, and the memory value MEM_CNT from the memory 34. The sign determination unit receives the count value which is the output of the counter 36. The memory 34 receives the control signal CNT_WR from the count control unit 40 and the count value with a predetermined sign attached by the sign inversion unit 38. The comparator 52 of the determination unit 50 receives the count value which is the output of the counter 36, the positive threshold value th_p, and the negative threshold value th_m.

[0053] Next, a method for driving the photoelectric conversion device according to the present embodiment will be described with reference to FIGS. 9 and 10. FIG. 10 shows the count value of the counter 36, the value stored in the memory 34, the event value which is the output of the determination unit 50, and the waveforms of the signal CLKB, the reset signal RES, and the control signals CNT_RES, CNT_WR, and CNT_RD.

[0054] First, the count value of the counter 36 is reset to 0 (step S201). At time t1s, when the reset signal RES is supplied from the control unit 70 to the count control unit 40, the count control unit 40 generates the control signal CNT_RES which is the same as the reset signal RES and supplies it to the counter 36. The counter 36 resets the count value to 0 using the control signal CNT_RES as a trigger.

[0055] Also at time t1s, it is assumed that the signal CLKB transitions from the low level to the high level and the exposure period starts. The counter 36 starts an up - count of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 (step S202).

[0056] At the subsequent time t1e, the count control unit 40 outputs a control signal CNT_WR in response to the falling edge (negative edge) of the signal CLKB. Also, the sign inversion unit 38 inverts the sign of the count value which is the output of the counter 36 and outputs it to the memory 34. The memory 34 stores, as a memory value MEM_CNT, the count value whose sign has been inverted by the sign inversion unit 38, triggered by the control signal CNT_WR (step S204). The sign inversion of the count value in the sign inversion unit 38 may convert the count value to the one's complement or the two's complement. The former can be implemented by a simple bit inversion, but an adder is required for the latter. Also, when an inverted output can be obtained from the counter 36, the inverted output of the counter 36 may be directly supplied to the memory 34 without using the sign inversion unit 38.

[0057] At the subsequent time t2s, the count control unit 40 outputs a control signal CNT_RD in response to the rising edge of the signal CLKB. The counter 36 reads the memory value MEM_CNT from the memory 34 triggered by the control signal CNT_RD, and updates its own count value with the read memory value MEM_CNT (step S205).

[0058] Also at the time t2s, the counter 36 starts an up - count of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the rising edge of the signal CLKB, starting from the updated count value (step S206). The up - count in the period d2 is performed from the count value obtained by inverting the sign of the count value in the period d1. As a result, as the count value at the time t2e, a difference value between the count value in the period d2 and the count value in the period d1 can be obtained.

[0059] At the subsequent time t2e, when the signal CLKB transitions from a high level to a low level, the determination unit 50 performs event determination processing based on the count value received from the counting unit 30 and the threshold values th_p and th_m (step S207). The determination unit 50 outputs, as an event value, the determination result based on the comparison result between the count value and the threshold values th_p and th_m to the count control unit 40.

[0060] The count control unit 40 controls the counter 36 according to the event value received from the determination unit 50. Specifically, when no event is detected, that is, when the event value is 0 (''NO'' in step S208), it returns to step S205 and updates the count value of the counter 36 with the memory value MEM_CNT read from the memory 34. In the driving example of FIG. 10, the event value is 0 at time t2e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t3s. Further, the event value is also 0 at the subsequent time t3e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t4s. The operations in the period d3 from time t3s to time t3e and the period d4 from time t4s to time t4e are the same as those in the period d2 from time t2s to time t2e.

[0061] When an event is detected, that is, when the event value is 1 or -1 (''YES'' in step S208), it proceeds to step S209. The count control unit 40 generates a control signal CNT_RES and supplies it to the counter 36. The counter 36 resets the count value to 0 using the control signal CNT_RES as a trigger. In the driving example of FIG. 10, since the count value is below the threshold value th_m at time t4e, the event value is -1, and the count value of the counter 36 is reset to 0 at the subsequent time t5s. After the reset of the counter 36, it returns to step S202. That is, after time t5s, the same processing as that from time t1s is performed.

[0062] At the subsequent time t5e, the count control unit 40 generates a control signal CNT_WR in response to the falling edge of the signal CLKB. Also, the sign inversion unit 38 inverts the sign of the count value which is the output of the counter 36, and outputs it to the memory 34. The memory 34 stores, as a memory value MEM_CNT, the count value whose sign has been inverted by the sign inversion unit 38, triggered by the high-level control signal CNT_WR (step S204).

[0063] At the subsequent time t6s, the count control unit 40 generates a control signal CNT_RD in response to the rising edge of the signal CLKB. The counter 36 reads out the memory value MEM_CNT from the memory 34 triggered by the control signal CNT_RD, and updates its own count value with the read memory value MEM_CNT (step S205).

[0064] Also at the time t6s, the counter 36 starts an up-count of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the rising edge of the signal CLKB, starting from the updated count value (step S206). The up-count in the period d6 is performed from the count value obtained by inverting the sign of the count value in the period d5. As a result, as the count value at the time t6e, a difference value between the count value in the period d6 and the count value in the period d5 can be obtained.

[0065] At the subsequent time t6e, when the signal CLKB transitions from the high level to the low level, the determination unit 50 performs event determination processing based on the count value received from the count unit 30 and the threshold values th_p, th_m (step S207). The determination unit 50 outputs, as an event value, the determination result based on the comparison result between the count value and the threshold values th_p, th_m, to the count control unit 40.

[0066] The count control unit 40 controls the counter 36 according to the event value received from the determination unit 50. Specifically, when no event is detected, that is, when the event value is 0 (''NO'' in step S208), it returns to step S205 and updates the count value of the counter 36 with the memory value MEM_CNT read from the memory 34. In the driving example of FIG. 10, the event value is 0 at time t6e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t7s. Further, the event value is also 0 at the subsequent time t7e, and the memory value MEM_CNT is read from the memory 34 at the subsequent time t8s. The operations in the period d7 from time t7s to time t7e and the period d8 from time t8s to time t8e are the same as those in the period d2 from time t2s to time t2e.

[0067] When an event is detected, that is, when the event value is 1 or -1 (''YES'' in step S208), it proceeds to step S209. The count control unit 40 generates a high-level control signal CNT_RES and supplies it to the counter 36. The counter 36 resets the count value to 0 using the high-level control signal CNT_RES as a trigger. In the driving example of FIG. 10, since the count value exceeds the threshold th_p at time t8e, the event value is 1, and the count value of the counter 36 is reset to 0 at the subsequent time t9s. After the counter 36 is reset, it returns to step S202. That is, after time t9s, the same processing as that from time t1s is performed.

[0068] As described above, in this embodiment, by performing the sign inversion process of the count value of the counter, the difference value between the two exposure periods can be obtained as the count value. Therefore, the subtractor for calculating the difference value can be reduced, and the configuration of the photoelectric conversion device can be simplified.

[0069] Therefore, according to this embodiment, it is possible to realize a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

[0070] [Third Embodiment] The photoelectric conversion device and its driving method according to the third embodiment of the present invention will be described with reference to FIGS. 11 to 14. FIG. 11 is a block diagram showing a configuration example of a count unit, a count control unit, and a determination unit of the photoelectric conversion device according to the present embodiment. FIG. 12 is a flowchart showing a driving method of the photoelectric conversion device according to the present embodiment. FIGS. 13 and 14 are timing diagrams showing the driving method of the photoelectric conversion device according to the present embodiment. The same components as those of the photoelectric conversion device according to the first or second embodiment are denoted by the same reference numerals, and the description thereof is omitted or simplified.

[0071] The photoelectric conversion device according to the present embodiment is the same as the photoelectric conversion device according to the first embodiment, except that the configuration of the count unit 30 is different. In the present embodiment, the description will be centered on the points different from the photoelectric conversion device according to the first embodiment, and the description of the same points as those of the photoelectric conversion device according to the first embodiment will be omitted as appropriate.

[0072] As shown in FIG. 11, the count unit 30 of the photoelectric conversion device according to the present embodiment is composed of a UD counter 32 capable of performing both up counting and down counting. The count unit 30 of the present embodiment does not have a memory 34 such as that provided in the count units 30 of the first and second embodiments.

[0073] The count control unit 40 receives the signal CLKB and the reset signal RES from the control unit 70, and the event value which is the output signal of the determination unit 50, and generates control signals CNT_RES and CNT_UD. The UD counter 32 receives the pulse signal from the photoelectric conversion unit 20 and the control signals CNT_RES and CNT_UD from the count control unit 40. The comparator 52 of the determination unit 50 receives the count value which is the output of the UD counter 32, the positive threshold value th_p, and the negative threshold value th_m.

[0074] In FIG. 11, the event value, which is the output signal of the determination unit 50, is configured to be fed back to the count control unit 40. However, when only the driving method described in this embodiment is implemented, the feedback of the event value is not necessarily required. The configuration example shown in FIG. 11 is considered for the case where the driving method of the fourth embodiment described later is also implemented.

[0075] Next, the driving method of the photoelectric conversion device according to this embodiment will be described with reference to FIGS. 12 to 14. FIGS. 13 and 14 show the count value of the counter 36, the event value which is the output of the determination unit 50, and the waveforms of the signal CLKB, the reset signal RES, and the control signals CNT_RES and CNT_UD.

[0076] First, when the reset signal RES is supplied from the control unit 70 to the count control unit 40 at time t1s, the count control unit 40 generates the control signal CNT_RES that is the same as the reset signal RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S301).

[0077] Also at time t1s, after receiving the reset signal RES, the count control unit 40 controls the control signal CNT_UD to the low level in response to the rising edge of the first signal CLKB. The UD counter 32 starts downcounting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the low-level control signal CNT_UD (step S302). The downcounting in the UD counter 32 continues until the time t1e when the signal CLKB next transitions from the high level to the low level.

[0078] At the subsequent time t2s, when the signal CLKB transitions from the low level to the high level again, the count control unit 40 controls the control signal CNT_UD from the low level to the high level in response to the rising edge of the signal CLKB. The UD counter 32 starts up-counting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the high-level control signal CNT_UD (step S303). The up-counting in the UD counter 32 starts from the count value at the time t1e and continues until the time t2e when the signal CLKB next transitions from the high level to the low level. As a result, as the count value at the time t2e, a difference value between the count value in the period d2 and the count value in the period d1 can be obtained.

[0079] At the subsequent time t2e, when the signal CLKB transitions from the high level to the low level, the determination unit 50 performs event determination processing based on the count value received from the count unit 30 and the threshold values th_p and th_m (step S304). The event determination result is output from the determination unit 50 as an event value.

[0080] At the subsequent time t3s, the count control unit 40 generates a control signal CNT_RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S305). After that, it returns to step S302, and the processing from step S302 to step S305 is repeated in the same manner as the processing from time t1s to time t2e.

[0081] As described above, in this embodiment, the count control unit 40 performs a toggle operation of switching the signal level of the control signal CNT_UD each time it detects the rising edge of the signal CLKB. That is, the count control unit 40 controls the UD counter to perform a downcount during the odd-numbered exposure periods, i.e., the periods d1, d3, d5, d7, d9,.... Also, during the even-numbered exposure periods, i.e., the periods d2, d4, d6, d8, ..., the count control unit 40 controls the UD counter 32 to perform an upcount. Before the periods d1, d3, d5, d7, d9, ... which are the odd-numbered exposure periods, the count control unit 40 generates the control signal CNT_RES and resets the count value of the UD counter 32 to 0 using this as a trigger. Then, after the elapse of the periods d2, d4, d6, d8, ... which are the even-numbered exposure periods, the event determination process in the determination unit 50 is performed.

[0082] The output of the event value according to the result of the event determination process is the same as in the first embodiment. That is, in the event determination, if the count value is greater than or equal to the threshold value th_m and less than or equal to the threshold value th_p, the event value 0 is output as no event detected. If the count value is less than the threshold value th_m, the event value -1 is output as an event detected. If the count value is greater than the threshold value th_p, the event value 1 is output as an event detected.

[0083] As described above, in this embodiment, regardless of the occurrence of an event, the downcount and upcount are always repeated, and the difference value between the two exposure periods can be obtained as the count value. Although the event detection frequency decreases compared to the first embodiment, in addition to the subtracter for calculating the difference value, the memory for storing the count value can be reduced. However, in this embodiment, since the event detection is performed by obtaining the difference value of the count values in two consecutive exposure periods, for example, when a certain luminance change below the event detection threshold continues as shown in the timing chart of FIG. 14, the event detection does not occur.

[0084] Therefore, according to the present embodiment, it is possible to realize a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

[0085] [Fourth Embodiment] A photoelectric conversion device and a driving method thereof according to a fourth embodiment of the present invention will be described with reference to FIGS. 15 and 16. The same components as those of the photoelectric conversion device according to the first to third embodiments are denoted by the same reference numerals, and the description thereof is omitted or simplified.

[0086] In the present embodiment, in the photoelectric conversion device according to the third embodiment, a method for improving an operation in which event detection is not performed when a certain luminance change below an event detection threshold continues is shown. The configurations of the count unit, the count control unit, and the determination unit in the photoelectric conversion device according to the present embodiment are the same as the configuration example of the third embodiment described with reference to FIG. 11.

[0087] FIG. 15 is a flowchart showing a driving method of the photoelectric conversion device according to the present embodiment. FIG. 16 is a timing chart showing a driving method of the photoelectric conversion device according to the present embodiment. FIG. 16 shows the count value of the counter 36, the event value which is the output of the determination unit 50, and the waveforms of the signal CLKB, the reset signal RES, and the control signals CNT_RES and CNT_UD.

[0088] First, when a reset signal RES is supplied from the control unit 70 to the count control unit 40 at time t1s, the count control unit 40 generates a control signal CNT_RES identical to the reset signal RES and supplies it to the up-down counter 32. The up-down counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S401).

[0089] Also at time t1s, after receiving the reset signal RES, the count control unit 40 controls the control signal CNT_UD to a low level in response to the rising edge of the first signal CLKB. The UD counter 32 starts downcounting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the low-level control signal CNT_UD (step S402). The downcounting in the UD counter 32 continues until the time t1e when the signal CLKB next transitions from a high level to a low level.

[0090] At the subsequent time t2s, when the signal CLKB transitions from a low level to a high level again, the count control unit 40 controls the control signal CNT_UD from a low level to a high level in response to the rising edge of the signal CLKB. The UD counter 32 starts upcounting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the high-level control signal CNT_UD (step S403). The upcounting in the UD counter 32 starts from the count value at time t1e and continues until the time t2e when the signal CLKB next transitions from a high level to a low level. As a result, as the count value at time t2e, the difference value between the count value in period d2 and the count value in period d1 can be obtained.

[0091] At the subsequent time t2e, when the signal CLKB transitions from a high level to a low level, the determination unit 50 performs event determination processing based on the count value received from the count unit 30 and the threshold values th_p and th_m (step S404). The event determination result is output from the determination unit 50 as an event value.

[0092] Also in this embodiment, similar to the third embodiment, the count control unit 40 performs a toggle operation of switching the signal level of the control signal CNT_UD each time it detects the rising edge of the signal CLKB. That is, the count control unit 40 controls the UD counter to perform a downcount in the odd-numbered exposure periods, i.e., the periods d1, d3, d5, d7, d9,.... Also, in the even-numbered exposure periods, i.e., the periods d2, d4, d6, d8,...., the UD counter 32 is controlled to perform an upcount. Then, after the elapse of the even-numbered exposure periods, i.e., the periods d2, d4, d6, d8,...., the event determination process in the determination unit 50 is performed.

[0093] However, in this embodiment, after the event determination process in the determination unit 50, the count value of the UD counter 32 is set according to the result. Specifically, when no event is detected, that is, when the event value is 0 (``NO'' in step S405), the process returns to step S402 without resetting the count value of the UD counter 32, and proceeds to the next exposure period. When an event is detected, that is, when the event value is 1 or -1 (``YES'' in step S405), the process proceeds to step S406. The count control unit 40 generates a control signal CNT_RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger. Then, the process returns to step 402 and proceeds to the next exposure period.

[0094] For example, in the driving example of FIG. 16, the event values in the event determination processes after each of the times t2e, t4e, and t8e are 0. Therefore, the exposure periods of the periods d3, d5, d9 start the count operation from the count value in the immediately preceding exposure period. In contrast, the event value in the event determination process after the time t6e is 1. Therefore, the exposure period of the period d7 starts the count operation from 0 after resetting the count value of the period d6.

[0095] That is, in the present embodiment, when no event occurs, the current count value, that is, the difference value of the count values in the previous exposure periods, is not reset, and the count operations in the next two exposure periods are continuously performed. Therefore, since the difference value can be accumulated when no event occurs, event detection is possible even when a certain luminance change below the event detection threshold continues.

[0096] In this way, in the present embodiment, regardless of the occurrence of an event, the downcount and upcount are always repeated, and the difference value between two exposure periods can be obtained as the counter value. Although the event detection frequency decreases compared to the first embodiment, the memory for storing the count value in addition to the subtracter for calculating the difference value can be reduced. Also, event detection is possible even when a certain luminance change below the event detection threshold, which was a problem in the third embodiment, continues.

[0097] Therefore, according to the present embodiment, a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale can be realized.

[0098] [Fifth Embodiment] The photoelectric conversion device and its driving method according to the fifth embodiment of the present invention will be described with reference to FIGS. 17 to 19. FIG. 17 is a block diagram showing a configuration example of the count unit, count control unit, and determination unit of the photoelectric conversion device according to the present embodiment. FIG. 18 is a flowchart showing the driving method of the photoelectric conversion device according to the present embodiment. FIG. 19 is a timing diagram showing the driving method of the photoelectric conversion device according to the present embodiment. The same components as those of the photoelectric conversion device according to the first to fourth embodiments are denoted by the same reference numerals, and the description is omitted or simplified.

[0099] The photoelectric conversion device according to the present embodiment is the same as the photoelectric conversion device according to the first embodiment, except that the configurations of the count unit 30 and the determination unit 50 are different. In the present embodiment, the description will focus on the points different from the photoelectric conversion device according to the first embodiment, and the description of the same points as those of the photoelectric conversion device according to the first embodiment will be omitted as appropriate.

[0100] As shown in FIG. 17, the counting unit 30 of the photoelectric conversion device according to this embodiment is composed of a selector SEL and a UD counter 32, and the determination unit 50 is composed of a sign determination unit 54.

[0101] The count control unit 40 receives the signal CLKB and the reset signal RES from the control unit 70, the count value which is the output signal of the UD counter 32, and the event value which is the output signal of the determination unit 50, and generates control signals CNT_SEL, CNT_RES, CNT_UD. Here, the control signal CNT_SEL is a signal for switching the output signal from the selector SEL. For example, when the control signal CNT_SEL is at a low level, the selector SEL selects and outputs the pulse signal from the photoelectric conversion unit 20, and when the control signal CNT_SEL is at a high level, the selector SEL selects and outputs a pulse signal (threshold pulse signal) with a number corresponding to a predetermined threshold value th. The output signal of the selector SEL and the control signals CNT_RES and CNT_UD from the count control unit 40 are input to the UD counter 32.

[0102] The event value which is the output signal of the determination unit 50 is generated based on the result of the determination in the sign determination unit 54. That is, the sign determination unit 54 determines the presence or absence of event occurrence based on the relationship between the sign of the count value which is the output signal from the counting unit 30 and the sign of the count value after adding or subtracting a predetermined threshold value. For example, the event value can be defined as in the following formula (2).

Equation

[0103] When the count value, which is the output signal from the counting unit 30, is a positive value, a predetermined threshold value th is subtracted from the count value (added with -th). When the subtracted count value (added value) is a positive value, that is, when the count value, which is the output signal from the counting unit 30, is greater than the threshold value th, it is determined as an on event, and the event value 1 is output. When the count value, which is the output signal from the counting unit 30, is a negative value, the predetermined threshold value th is added to the count value. When the added count value (added value) is a negative value, that is, when the absolute value of the count value, which is the output signal from the counting unit 30, is greater than the threshold value th, it is determined as an off event, and the event value -1 is output. In other cases, that is, when the sign of the count value (added value) obtained by adding the count value by the threshold value th is inverted from the sign of the count value, which is the output signal from the counting unit 30, it is determined that no event has occurred, and the event value 0 is output. Note that since the determination of the sign of the count value only needs to monitor the negative sign bit (1 bit), it can be realized by a small-scale circuit.

[0104] In the first to fourth embodiments, since a comparator is used for event determination, a subtraction circuit, a flip-flop for data holding, signal lines for supplying threshold data, etc. are required, and there is a concern that the circuit scale and wiring area will increase. On the other hand, in this embodiment, since a comparator is not used for the event determination process, it is possible to simplify the processing circuit of the determination unit 50 and reduce the circuit scale.

[0105] Next, a method for driving the photoelectric conversion device according to this embodiment will be described with reference to FIGS. 18 and 19. FIG. 19 shows the waveforms of the count value of the UD counter 32, the event value which is the output of the determination unit 50, and the signals CLKB, reset signal RES, threshold pulse signal, and control signals CNT_RES, CNT_UD, CNT_SEL.

[0106] In the driving method of this embodiment, similar to the third and fourth embodiments, the count control unit 40 performs a toggle operation of switching the signal level of the control signal CNT_UD every time it detects the rising edge of the signal CLKB. That is, the count control unit 40 controls the UD counter to perform a downcount in the odd-numbered exposure periods, such as periods d1, d3, d5, d7, d9, …. Also, in the even-numbered exposure periods, such as periods d2, d4, d6, d8, …, the UD counter 32 is controlled to perform an upcount. Then, after the elapse of the even-numbered exposure periods, such as periods d2, d4, d6, d8, …, the event determination process in the determination unit 50 is performed.

[0107] The control signal CNT_SEL is controlled to be at a high level during the periods dth1, dth2, dth3, dth4, … in which the determination process is performed, and is controlled to be at a low level during other periods. In each of the periods dth1, dth2, dth3, dth4, …, a number of threshold pulse signals corresponding to the threshold value th are input to the UD counter 32 via the selector SEL in response to the high-level control signal CNT_SEL.

[0108] First, when a reset signal RES is supplied from the control unit 70 to the count control unit 40 at time t1s, the count control unit 40 generates a control signal CNT_RES identical to the reset signal RES and supplies it to the UD counter 32. The UD counter 32 resets its count value to 0 using the control signal CNT_RES as a trigger (step S501).

[0109] Also at time t1s, after receiving the reset signal RES, the count control unit 40 controls the control signal CNT_UD to be at a low level in response to the first rising edge of the signal CLKB. The UD counter 32 starts a downcount of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the low-level control signal CNT_UD (step S502). The downcount in the UD counter 32 continues until the time t1e when the signal CLKB next transitions from a high level to a low level.

[0110] At the subsequent time t2s, when the signal CLKB transitions from the low level to the high level again, the count control unit 40 controls the control signal CNT_UD from the low level to the high level in response to the rising edge of the signal CLKB. The UD counter 32 starts an up - count of the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the high - level control signal CNT_UD (step S503). The up - count in the UD counter 32 starts from the count value at the time t1e and continues until the time t2e when the signal CLKB transitions from the high level to the low level next. As a result, as the count value at the time t2e, a difference value between the count value in the period d2 and the count value in the period d1 can be obtained.

[0111] At the subsequent time t2e, when the signal CLKB transitions to the low level and the control signal CNT_SEL transitions to the high level, a determination period dth1 starts. In the determination period, an additional count by a predetermined threshold value is performed on the count value at the time when the immediately preceding exposure period (period d2) ended (step S504). The count control unit 40 sets the control signal CNT_UD according to the sign of the count value at the time when the immediately preceding exposure period (period d2) ended. For example, when the count value in the immediately preceding exposure period is a negative value, the control signal CNT_UD is set to the high level (up - count), and when the count value in the immediately preceding exposure period is a positive value, the control signal CNT_UD is set to the low level (down - count). For example, in the case of the driving example in FIG. 19, since the count value at the time t2e, which is the end time of the period d2, is a negative value, the control signal CNT_UD becomes the high level, and an additional up - count by the threshold value th is performed from the count value at the time t2e.

[0112] The symbol determination unit 54 performs event determination processing based on the relationship between the sign of the count value at the time when the immediately preceding exposure period (period d2) ends and the sign of the count value after additional counting by the threshold value (step S505). Specifically, if the sign of the count value after additional counting by the threshold value has been inverted from the sign of the count value during the immediately preceding exposure period, it is determined that there is no event, and 0 is output as the event value. If the count value after additional counting by the threshold value maintains the same negative value as the count value during the immediately preceding exposure period, it is determined as an off-event, and -1 is output as the event value. If the count value after additional counting by the threshold value maintains the same positive value as the count value during the immediately preceding exposure period, it is determined as an on-event, and 1 is output as the event value.

[0113] At the subsequent time t3s, the count control unit 40 generates a control signal CNT_RES and supplies it to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S506). After that, the process returns to step S502, and the processes from step S502 to step S506 are repeated in the same manner as the processes from time t1s to time t3s.

[0114] In the case of the driving example in FIG. 19, additional up-counting is executed during periods dth1 and dth2 according to the negative count value during the immediately preceding exposure period, and additional down-counting is executed during periods dth3 and dth4 according to the positive count value during the immediately preceding exposure period. In the determination processing during periods dth1 and dth3, since the sign of the count value has been inverted, it is determined that there is no event, and 0 is output as the event value. In the determination processing during period dth2, since the count value maintains a negative value, it is determined as an off-event, and -1 is output as the event value. In the determination processing during period dth4, since the count value maintains a positive value, it is determined as an on-event, and 1 is output as the event value.

[0115] Thus, in this embodiment, the presence or absence of an event occurrence can be determined simply by adding and subtracting a threshold value to a counter process and checking the change in the sign of the counter value. Therefore, although the event detection frequency decreases by the time required for the counter process of adding and subtracting the threshold value, the arithmetic circuit required for event determination can be reduced.

[0116] Therefore, according to this embodiment, it is possible to realize a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

[0117] [Sixth Embodiment] The photoelectric conversion device and its driving method according to the sixth embodiment of the present invention will be described with reference to FIGS. 20 to 23. FIG. 20 is a circuit diagram showing a configuration example of a photoelectric conversion unit of the photoelectric conversion device according to this embodiment. FIG. 21 is a block diagram showing a configuration example of a count unit, a count control unit, and a determination unit of the photoelectric conversion device according to this embodiment. FIG. 22 is a flowchart showing a driving method of the photoelectric conversion device according to this embodiment. FIG. 23 is a timing diagram showing a driving method of the photoelectric conversion device according to this embodiment. The same components as those of the photoelectric conversion device according to the first to fifth embodiments are denoted by the same reference numerals, and the description thereof is omitted or simplified.

[0118] In the first to fifth embodiments, it is assumed that the exposure periods of all the pixel units 12 are constant. For example, when the subject is dark (during shooting in a dark scene), the sensitivity is improved by increasing the period of a synchronization signal (signal CLKB) that determines the exposure period. On the other hand, when the subject is bright (during shooting in a bright scene), saturation (overflow of the counter) is avoided by shortening the period of signal CLKB. However, when a dark scene and a bright scene are mixed, in order to image both of them suitably, it is required to increase the period of signal CLKB in accordance with the dark scene and ensure a sufficient bit width of the counter so as not to saturate in the bright scene. Therefore, the circuit scale increases by the bit width of the counter.

[0119] In this embodiment, a configuration of a photoelectric conversion device and a driving method thereof capable of suitably imaging both a dark scene and a bright scene even when the bit width of a counter is small are shown.

[0120] The photoelectric conversion device according to this embodiment is the same as the photoelectric conversion device according to the third embodiment except that the configurations of a photoelectric conversion unit 20, a count unit 30, a count control unit 40, and a determination unit 50 are different. In this embodiment, the description will be centered on the differences from the photoelectric conversion device according to the first embodiment, and the description of the same points as those of the photoelectric conversion device according to the third embodiment will be omitted as appropriate.

[0121] As shown in FIG. 20, the photoelectric conversion unit 20 of the photoelectric conversion device according to this embodiment further includes a mask unit 28 in addition to a photoelectric conversion element 22, a switch unit 24, and a waveform shaping unit 26. The photoelectric conversion element 22 can be constituted by an APD. The switch unit 24 can be constituted by, for example, an N-type MOS transistor. The waveform shaping unit 26 can be constituted by a logic circuit such as an inverter circuit. The mask unit 28 is a logic circuit that outputs a signal according to a signal CLKB and a control signal CNT_PD. Note that the control signal CNT_PD is an output signal of the count control unit 40.

[0122] The anode of the APD constituting the photoelectric conversion element 22 is connected to a ground voltage node. The cathode of the APD constituting the photoelectric conversion element 22 is connected to the source of the N-type MOS transistor constituting the switch unit 24. The drain of the N-type MOS transistor constituting the switch unit 24 is connected to a node to which a voltage Vbias is supplied. The input node of the waveform shaping unit 26 is connected to a connection node (node A) between the photoelectric conversion element 22 and the switch unit 24. The output node (node B) of the waveform shaping unit 26 becomes the output node of the photoelectric conversion unit 20. The output signal of the mask unit 28 is input to the gate of the N-type MOS transistor constituting the switch unit 24.

[0123] The mask unit 28 is a logic circuit that outputs a high-level signal when the signal CLKB and the control signal CNT_PD are at a high level, and outputs a low-level signal when at least one of the signal CLKB and the control signal CNT_PD is at a low level. That is, the mask unit 28 supplies the result of the logical product of the signal CLKB and the control signal CNT_PD to the gate of the N-type MOS transistor constituting the switch unit 24.

[0124] The operation of the photoelectric conversion unit 20 when a single photon is incident is described below. When the signal CLKB and the control signal CNT_PD become high level and the switch unit 24 is turned on, the voltage Vbias is applied to the anode of the APD via the switch unit 24, and the node A becomes high level. When a photon is incident on the APD in this state, electron-hole pairs are generated by photoelectric conversion, and an avalanche multiplication phenomenon occurs using these charges as seeds. When an avalanche current flows through the APD due to the avalanche multiplication phenomenon, a voltage drop occurs due to the resistance component (quenching resistance) of the switch unit 24, and the bias voltage applied to the APD begins to drop. When the bias voltage drops to the breakdown voltage, the avalanche multiplication phenomenon stops, and the node A becomes low level. As a result, the avalanche current no longer flows through the APD, and the voltage Vbias is applied again to the cathode of the APD, and the node A returns to high level.

[0125] As a result, the voltage of the node A transitions from high level to low level as photons are incident on the APD, and then transitions from low level to high level. Thereby, the output signal of the waveform shaping unit 26, that is, the voltage of the node B, transitions from low level to high level in response to the change in the signal level of the node A, and then transitions from high level to low level. The pulse signal output in this way is a photon detection signal indicating that photons have been incident on the photoelectric conversion element 22.

[0126] Next, a configuration example of the count unit 30, the count control unit 40, and the determination unit 50 that constitute the pixel unit 12 will be described with reference to FIG. 21. The count unit 30 has, for example, a UD counter 32 capable of performing both up counting and down counting as shown in FIG. 21. The determination unit 50 has a comparator 52.

[0127] The count control unit 40 receives a signal CLKB from the control unit 70, control signals ST and STD which are timing signals for switching the count direction, a count value from the count unit 30, and an event value from the determination unit 50. Based on these signals, the count control unit 40 generates a control signal CNT_PD for the photoelectric conversion unit 20 and control signals CNT_RES and CNT_UD for the UD counter 32. Here, the control signal CNT_PD is a signal for stopping (power saving) the imaging operation in the photoelectric conversion unit 20. For example, the photoelectric conversion unit 20 operates when the control signal CNT_PD is at a high level and stops when the control signal CNT_PD is at a low level.

[0128] Next, a driving method of the photoelectric conversion device according to the present embodiment will be described with reference to FIGS. 22 and 23. FIG. 23 shows waveforms of the count value, the maximum value, and the minimum value of the UD counter 32, the event value which is the output of the determination unit 50, and signals CLKB and control signals ST, ST_F, CNT_RES, CNT_UD, and CNT_PD.

[0129] In the driving method of this embodiment, each of the periods during which the signal CLKB becomes high continuously (periods d12, d34, d56 in FIG. 23) includes two periods: an exposure period for performing downcount and an exposure period for performing upcount. And each process during the period when the signal CLKB is high is executed in synchronization with the control signal ST. During the period when the signal CLKB is high, the high-level control signal ST is output, for example, 5 times. The first control signal ST corresponds to the rising edge of the signal CLKB, and the last (fifth) control signal ST corresponds to the falling edge of the signal CLKB. The intervals at which the control signal ST is output during the period when the signal CLKB is high are set to increase sequentially by a power of 2. In other words, the plurality of periods divided by the control signal ST constitute a plurality of sub-periods whose lengths increase sequentially by a power of 2. For example, in the period d12 of FIG. 23, the period d12_2 is 2 to the power of 0 times (= 1 time) that of the period d12_1, the period d12_3 is 2 to the power of 1 times (= 2 times) that of the period d12_2, and the period d12_4 is 2 to the power of 2 times (= 4 times) that of the period d12_3. The control signal ST_F is output in synchronization with the second-to-last (fourth) control signal ST during the period when the signal CLKB is high.

[0130] Note that in the driving method of this embodiment, the exposure period for performing downcount and the exposure period for performing upcount can be set independently for each of the plurality of pixel portions 12 constituting the pixel array portion 10.

[0131] First, at time t12s, in response to the rising edge of the signal CLKB, the count control unit 40 outputs the control signal CNT_RES to the UD counter 32. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S601).

[0132] Also at time t12s, the count control unit 40 controls the control signal CNT_UD to a low level in response to the rising edge of the signal CLKB. The UD counter 32 starts downcounting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 in response to the low-level control signal CNT_UD (step S602). At this time, the control signal CNT_PD is at a high level, and the photoelectric conversion unit 20 is in a state where it can output a pulse signal.

[0133] Next, at time t12_2 when the control signal ST is received, the count control unit 40 determines whether the count value of the UD counter 32 is less than a predetermined value (-th_r), or whether the control signal ST_F is at a high level (step S603). As a result of the determination, if the count value of the UD counter 32 is greater than or equal to the predetermined value (-th_r) and the control signal ST_F is at a low level (''NO'' in step S603), the process returns to step S602 to continue the downcounting. If the count value of the UD counter 32 is less than the predetermined value (-th_r), or if the control signal ST_F is at a high level (''YES'' in step S603), the process proceeds to step S604. In the driving example of FIG. 23, since the count value of the UD counter 32 at time t12_2 is less than the predetermined value (-th_r), the process proceeds to step S604.

[0134] Here, the predetermined value th_r used as the determination criterion can be set to, for example, a value that is half of the maximum count value of the UD counter 32. That is, if the absolute value of the current count value does not exceed half of the maximum count value, the downcounting continues, and if the absolute value of the current count value exceeds half of the maximum count value, the process proceeds to step S604 to switch the count direction. In this way, each pixel unit 12 determines the timing to switch from downcounting to upcounting according to the count value of its own UD counter 32.

[0135] Note that by setting the predetermined value th_r to half of the maximum count value, the determination of whether it is exceeded only needs to monitor the most significant bit excluding the sign bit of the count value. That is, if the most significant bit of the count value is 1, it can be determined that the absolute value of the count value is greater than or equal to half of the maximum count value of the UD counter 32.

[0136] At the subsequent time t12_2, the count control unit 40 controls the control signal CNT_UD from the low level to the high level according to the count value of the UD counter 32 and the control signal ST. The UD counter 32 starts up-counting the pulse signal (photon detection signal) output from the photoelectric conversion unit 20 according to the high-level control signal CNT_UD (step S604). The up-counting in the UD counter 32 starts from the count value at time t12_2 and continues until the time t12_3 when the control signal ST is received next. As a result, as the count value at time t12_3, the difference value between the count value in the period d12_2 and the count value in the period d12_1 can be obtained. Note that the lengths of the period d12_1 and the period d12_2 are the same.

[0137] At the subsequent time t12_3, the count control unit 40 controls the control signal CNT_PD from the high level to the low level according to the control signal ST. The output signal of the mask unit 28 becomes low level in response to the low-level control signal CNT_PD, and the N-type MOS transistor of the switch unit 24 is turned off. Thereby, the output of the pulse signal (photon detection signal) from the photoelectric conversion unit 20 can be stopped, and the consumption of wasted power can be reduced (step S605: power save process). Even after the control signal CNT_PD transitions to the low level, the count value at time t12_3 is held in the UD counter 32.

[0138] At the subsequent time t12e, the signal CLKB transitions from a high level to a low level, and the exposure period ends. The determination unit 50 performs event determination processing based on the count value of the UD counter 32 and the threshold values th_p and th_m in response to the falling edge of the signal CLKB (step S606). The event determination result is output as an event value to the outside of the pixel unit 12 and the count control unit 40. Also, the count control unit 40 controls the control signal CNT_PD from a low level to a high level in response to the falling edge of the signal CLKB.

[0139] At the subsequent time t34s, the count control unit 40 outputs a high-level control signal CNT_RES to the UD counter 32 in response to the rising edge of the signal CLKB. The UD counter 32 resets the count value to 0 using the control signal CNT_RES as a trigger (step S607). After that, it returns to step S602, and for periods d34, d56, …, the processing from step S602 to step S607 is sequentially repeated in the same manner as the processing in period d12.

[0140] In period d34, downcounting is performed in periods d34_1 and d34_2, and upcounting is performed in period d34_3. As a result, the difference value between the count value in period d34_3 and the count values in periods d34_1 and d34_2 is obtained as the count value at the time point of t34_4. Note that the total length of periods d34_1 and d34_2 is the same as the length of period d34_3.

[0141] During period d56, downcounting is performed in periods d56_1, d56_2, and d56_3. At time t56_4, which is the end time of period d56_3, the count value of up / down counter 32 is greater than a predetermined value (-th_r), but since control signal ST_F is supplied (ST_F = 1), the downcounting is forcibly terminated and upcounting is started. As a result, the difference value between the count value in period d56_4 and the count values in periods d56_1, d56_2, and d56_3 is obtained as the count value at time t56e. Note that the total length of periods d34_1 and d34_2 is the same as the length of period d34_3. Note also that the total length of periods d56_1, d56_2, and d56_3 is the same as the length of period d56_4.

[0142] In this way, by controlling downcounting and upcounting using control signal ST whose interval changes in multiples of two to prevent overflow of the count value, it is possible to align the lengths of the downcounting period and the upcounting period and obtain a difference value.

[0143] Thus, in this embodiment, downcounting and upcounting are always repeated regardless of the occurrence of an event, and the count value is monitored for each pixel section 12 to control so as not to overflow. Thereby, it becomes possible to reduce the bit width of up / down counter 32.

[0144] Even if the above-described driving method is used, it is not always possible to prevent an overflow of the count value. For example, in period d34, at time t34_2, since the count value does not fall below the predetermined value (-th_r), the downcount continues for period d34_2 as well. However, if the light amount increases significantly during period d34_2, an overflow of the count value can also occur. As a measure to avoid this, for example, the setting of the predetermined value (th_r) with respect to the maximum count value can be made smaller, for example, set to 1 / 4 of the maximum count value. Thereby, the probability of the count value overflowing can be reduced. Alternatively, since an overflow occurs due to a large change in the light amount, an overflow may be detected and detected as an event.

[0145] Therefore, according to the present embodiment, it is possible to realize a photoelectric conversion device capable of high-resolution event detection while suppressing the circuit scale.

[0146] [Embodiment 7] The photoelectric conversion system according to the seventh embodiment of the present invention will be described with reference to FIG. 25. FIG. 25 is a block diagram showing a schematic configuration of the photoelectric conversion system according to the present embodiment.

[0147] The photoelectric conversion device 100 described in the above first to sixth embodiments is applicable to various photoelectric conversion systems. Examples of applicable photoelectric conversion systems include digital still cameras, digital camcorders, surveillance cameras, copiers, fax machines, mobile phones, in-vehicle cameras, observation satellites, and the like. Also, a camera module including an optical system such as a lens and an imaging device is included in the photoelectric conversion system. FIG. 25 illustrates a block diagram of a digital still camera as an example of these.

[0148] The photoelectric conversion system 200 illustrated in FIG. 25 includes an imaging device 201, a lens 202 that forms an optical image of a subject on the imaging device 201, a diaphragm 204 for variably controlling the amount of light passing through the lens 202, and a barrier 206 for protecting the lens 202. The lens 202 and the diaphragm 204 are an optical system that condenses light onto the imaging device 201. The imaging device 201 is the photoelectric conversion device 100 described in any of the first to sixth embodiments, and converts the optical image formed by the lens 202 into image data.

[0149] The photoelectric conversion system 200 also includes a signal processing unit 208 that processes the output signal output from the imaging device 201. The signal processing unit 208 generates image data from the digital signal output by the imaging device 201. Further, the signal processing unit 208 performs operations of outputting image data by performing various corrections and compressions as necessary. The imaging device 201 may include an AD conversion unit that generates the digital signal processed by the signal processing unit 208. The AD conversion unit may be formed on the semiconductor layer (semiconductor substrate) on which the photoelectric conversion unit of the imaging device 201 is formed, or may be formed on a semiconductor layer (semiconductor substrate) different from the semiconductor layer (semiconductor substrate) on which the photoelectric conversion unit of the imaging device 201 is formed. Further, the signal processing unit 208 may be formed on the same semiconductor layer (semiconductor substrate) as the imaging device 201.

[0150] The photoelectric conversion system 200 further includes a memory unit 210 for temporarily storing image data, and an external interface unit (external I / F unit) 212 for communicating with an external computer or the like. Further, the photoelectric conversion system 200 includes a recording medium 214 such as a semiconductor memory for recording or reading out imaging data, and a recording medium control interface unit (recording medium control I / F unit) 216 for recording or reading out to / from the recording medium 214. Note that the recording medium 214 may be built into the photoelectric conversion system 200 or may be detachable.

[0151] Furthermore, the photoelectric conversion system 200 includes an overall control and arithmetic unit 218 that performs various operations and controls the entire digital still camera, and a timing generation unit 220 that outputs various timing signals to the imaging device 201 and the signal processing unit 208. Here, the timing signals and the like may be input from the outside, and the photoelectric conversion system 200 may have at least the imaging device 201 and the signal processing unit 208 that processes the output signal output from the imaging device 201.

[0152] The imaging device 201 outputs an imaging signal to the signal processing unit 208. The signal processing unit 208 performs predetermined signal processing on the imaging signal output from the imaging device 201 and outputs image data. The signal processing unit 208 generates an image using the imaging signal.

[0153] Thus, according to this embodiment, a photoelectric conversion system to which the photoelectric conversion device 100 according to the first to sixth embodiments is applied can be realized.

[0154] [Eighth Embodiment] The photoelectric conversion system and the moving body according to the eighth embodiment of the present invention will be described with reference to FIG. 26. FIG. 26 is a diagram showing the configuration of the photoelectric conversion system and the moving body according to this embodiment.

[0155] FIG. 26(a) shows an example of a photoelectric conversion system related to an in-vehicle camera. The photoelectric conversion system 300 includes an imaging device 310. The imaging device 310 is the photoelectric conversion device 100 described in any one of the first to sixth embodiments. The photoelectric conversion system 300 includes an image processing unit 312 that performs image processing on a plurality of image data acquired by the imaging device 310, and a parallax acquisition unit 314 that calculates a parallax (phase difference of a parallax image) from the plurality of image data acquired by the photoelectric conversion system 300. Further, the photoelectric conversion system 300 includes a distance acquisition unit 316 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 318 that determines whether there is a possibility of collision based on the calculated distance. Here, the parallax acquisition unit 314 and the distance acquisition unit 316 are examples of distance information acquisition means for acquiring distance information to an object. That is, the distance information is information related to parallax, defocus amount, distance to an object, and the like. The collision determination unit 318 may determine the possibility of collision using any of these distance information. The distance information acquisition means may be realized by dedicatedly designed hardware, or may be realized by a software module. Further, it may be realized by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated circuit), or the like, or may be realized by a combination of these.

[0156] The photoelectric conversion system 300 is connected to a vehicle information acquisition device 320 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Further, the photoelectric conversion system 300 is connected to a control ECU 330, which is a control device that outputs a control signal for generating a braking force for the vehicle based on the determination result of the collision determination unit 318. Further, the photoelectric conversion system 300 is also connected to an alarm device 340 that issues an alarm to the driver based on the determination result of the collision determination unit 318. For example, when the collision determination unit 318 determines that there is a high possibility of a collision, the control ECU 330 performs vehicle control to avoid the collision and reduce damage, such as applying the brakes, returning the accelerator, and suppressing the engine output. The alarm device 340 warns the user by sounding an alarm such as a sound, displaying alarm information on a screen of a car navigation system, or applying vibration to the seat belt or steering wheel.

[0157] In the present embodiment, the photoelectric conversion system 300 images the surroundings of the vehicle, for example, the front or the rear. FIG. 26(b) shows the photoelectric conversion system when imaging the front of the vehicle (imaging range 350). The vehicle information acquisition device 320 sends an instruction to the photoelectric conversion system 300 or the imaging device 310. With such a configuration, the ranging accuracy can be further improved.

[0158] In the above, an example of controlling so as not to collide with other vehicles has been described, but it is also applicable to control for automatically driving following other vehicles and control for automatically driving so as not to deviate from the lane. Further, the photoelectric conversion system is not limited to vehicles such as the host vehicle, and can be applied to moving bodies (moving devices) such as ships, aircraft, or industrial robots. In addition, it can be applied not only to moving bodies but also to devices that widely use object recognition, such as an advanced road traffic system (ITS).

[0159] [Embodiment 9] The device according to the ninth embodiment of the present invention will be described with reference to FIG. 27. FIG. 27 is a block diagram showing a schematic configuration of the device according to the present embodiment.

[0160] FIG. 27 is a schematic diagram showing a device EQP including a photoelectric conversion device APR. The photoelectric conversion device APR has the functions of the photoelectric conversion device 100 according to any one of the first to sixth embodiments. All or part of the photoelectric conversion device APR is a semiconductor device IC. The photoelectric conversion device APR in this example can be used, for example, as an image sensor, an AF (Auto Focus) sensor, a photometric sensor, or a distance measurement sensor. The semiconductor device IC has a pixel area PX in which pixel circuits PXC including photoelectric conversion units are arranged in a matrix. The semiconductor device IC can have a peripheral area PR around the pixel area PX. Circuits other than the pixel circuits can be arranged in the peripheral area PR.

[0161] The photoelectric conversion device APR may have a structure (chip stacking structure) in which a first semiconductor chip provided with a plurality of photoelectric conversion units and a second semiconductor chip provided with peripheral circuits are stacked. The peripheral circuits in the second semiconductor chip can be column circuits corresponding to the pixel columns of the first semiconductor chip, respectively. Also, the peripheral circuits in the second semiconductor chip can be matrix circuits corresponding to the pixels or pixel blocks of the first semiconductor chip, respectively. The connection between the first semiconductor chip and the second semiconductor chip can employ through electrodes (TSV), inter-chip wiring by direct bonding of a conductor such as copper, connection by micro bumps between chips, connection by wire bonding, and the like.

[0162] In addition to the semiconductor device IC, the photoelectric conversion device APR may include a package PKG that houses the semiconductor device IC. The package PKG can include a base on which the semiconductor device IC is fixed, a lid such as glass facing the semiconductor device IC, and connection members such as bonding wires and bumps that connect the terminals provided on the base and the terminals provided on the semiconductor device IC.

[0163] The equipment EQP may further include at least one of an optical device OPT, a control device CTRL, a processing device PRCS, a display device DSPL, a memory device MMRY, and a mechanical device MCHN. The optical device OPT corresponds to a photoelectric conversion device APR as a photoelectric conversion device, and is, for example, a lens, a shutter, or a mirror. The control device CTRL controls the photoelectric conversion device APR, and is, for example, a semiconductor device such as an ASIC. The processing device PRCS processes the signal output from the photoelectric conversion device APR, and constitutes an AFE (analog front end) or a DFE (digital front end). The processing device PRCS is a semiconductor device such as a CPU (central processing unit) or an ASIC (application-specific integrated circuit). The display device DSPL is an EL display device or a liquid crystal display device that displays the information (image) obtained by the photoelectric conversion device APR. The memory device MMRY is a magnetic device or a semiconductor device that stores the information (image) obtained by the photoelectric conversion device APR. The memory device MMRY is a volatile memory such as an SRAM or a DRAM, or a non-volatile memory such as a flash memory or a hard disk drive. The mechanical device MCHN has a movable part or a propulsion part such as a motor or an engine. In the equipment EQP, the signal output from the photoelectric conversion device APR is displayed on the display device DSPL or transmitted to the outside by a communication device (not shown) provided in the equipment EQP. For this purpose, it is preferable that the equipment EQP further includes a memory device MMRY and a processing device PRCS separately from the memory circuit part and the arithmetic circuit part of the photoelectric conversion device APR.

[0164] The equipment EQP shown in FIG. 27 may be an electronic device such as an information terminal having a photographing function (for example, a smartphone or a wearable terminal) or a camera (for example, a single-lens reflex camera, a compact camera, a video camera, a surveillance camera). The mechanical device MCHN in the camera can drive the components of the optical device OPT for zooming, focusing, and shutter operation. Further, the equipment EQP may be a transportation device (mobile body) such as a vehicle, a ship, or an aircraft. Further, the equipment EQP may be a medical device such as an endoscope or a CT scanner.

[0165] The mechanical device MCHN in the transport device can be used as a moving device. The device EQP as a transport device is suitable for transporting the photoelectric conversion device APR or for assisting and / or automating driving (operation) by means of a photographing function. The processing device PRCS for assisting and / or automating driving (operation) can perform processing for operating the mechanical device MCHN as a moving device based on the information obtained by the photoelectric conversion device APR.

[0166] The photoelectric conversion device APR according to the present embodiment can provide high value to its designer, manufacturer, seller, purchaser, and / or user. Therefore, if the photoelectric conversion device APR is mounted on the device EQP, the value of the device EQP can also be increased. Thus, in manufacturing and selling the device EQP, determining to mount the photoelectric conversion device APR of the present embodiment on the device EQP is advantageous for increasing the value of the device EQP.

[0167] [Modified Embodiment] The present invention is not limited to the above embodiments and various modifications are possible.

[0168] For example, an example in which a part of the configuration of any one of the embodiments is added to another embodiment or an example in which a part of the configuration of another embodiment is replaced is also an embodiment of the present invention. For example, the configuration of the first or second embodiment in which the counting unit 30 includes the memory 34 can be combined with the configuration of the fifth embodiment in which the circuit configuration of the determination unit 50 is simplified.

[0169] Also, in the above embodiment, a method of performing an up-count following a down-count is exemplified, but it is also possible to perform a down-count following an up-count. That is, in the first period of the exposure period, one of the up-count and the down-count may be performed, and in the next period of the exposure period, the other of the up-count and the down-count may be performed. In this case, the sign of the event determination may be reversed from that of the above embodiment. Similarly, in the second embodiment, an example using the counter 36 that performs an up-count is shown, but a counter 36 that performs a down-count can also be used.

[0170] In addition, in the sixth embodiment described above, the exposure period determined by the high-level period of the signal CLKB is divided into four periods by the control signal ST. However, the number of divisions of the exposure period is not limited to four, and it may be three or five or more.

[0171] In addition, in the above embodiment, it is assumed that all the pixel units 12 constituting the pixel array unit 10 are pixels for the purpose of only event detection. However, a configuration for simultaneously acquiring frame information (count value information proportional to the amount of light) can also be easily realized. For example, as shown in FIG. 24, the configuration can be such that the count value can be output from the count unit 30 to the readout unit 60 without passing through the determination unit 50.

[0172] However, in the above embodiment, since there is a timing when the counter shows the difference value between the count values of two exposure periods as the count value, in order to output the frame information, it is necessary to devise a method of outputting the count value based on only the count period in one direction. For example, in the first embodiment, the count value stored in the memory 34 can be output as the frame count value. In addition, in other embodiments, the count value in the exposure period immediately after the count value is reset may be output as the frame information.

[0173] In addition, when an event pixel that outputs event information and a frame pixel that outputs frame information coexist, it can be arbitrarily selected which pixel among the event pixel, the frame pixel, and the shared pixel each pixel unit 12 constituting the pixel array unit 10 becomes. For example, all the pixel units 12 constituting the pixel array unit 10 can be shared pixels. Also, for example, in the case of the color filter array in FIG. 2(c), the R pixel, the G pixel, and the B pixel can be set as frame pixels, and the C pixel can be set as an event pixel.

[0174] In addition, the imaging systems described in the seventh and eighth embodiments above are examples of imaging systems to which the photoelectric conversion device of the present invention can be applied, and the imaging systems to which the photoelectric conversion device of the present invention can be applied are not limited to the configurations shown in FIGS. 25 and 26(a).

[0175] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0176] Note that the above embodiments are merely examples of concretizations in implementing the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0177] The disclosure of the above embodiments includes the following configurations. (Configuration 1) A photoelectric conversion unit that outputs a pulse signal in response to the incidence of light, A counting unit having a counter that counts the pulse signal output from the photoelectric conversion unit during a predetermined exposure period, A count control unit that controls the counting unit, A pixel unit having a determination unit that performs a determination process of determining the presence or absence of a change in the amount of light incident on the photoelectric conversion unit based on the count value of the counter, The exposure period has a plurality of periods including at least a first period and a second period, The count control unit controls the counting unit to output a difference value between a first count value in the first period and a second count value in the second period, The determination unit performs the determination process based on the difference value A photoelectric conversion device characterized by the above. (Configuration 2) The counter can perform up - counting and down - counting, and the count control unit, controls the counter to perform either up - counting or down - counting during the first period, and during the second period, controls the counter to perform the other of the up - counting and the down - counting, and starts the counting operation from the first count value to generate the difference value. The photoelectric conversion device according to Configuration 1, characterized in that. (Configuration 3) The counting unit further includes a count value holding unit that holds the first count value, and when the difference value is equal to or less than a predetermined threshold value, the count control unit reads the first count value held by the count value holding unit into the counter, and controls the counter to start the counting operation during the third period of the exposure period from the first count value. The determination unit further performs the determination process based on the difference value between the first count value and the third count value during the third period. The photoelectric conversion device according to Configuration 1 or 2, characterized in that. (Configuration 4) The photoelectric conversion device further includes a sign inversion unit that inverts the sign of the count value output from the counting unit, and the count control unit generates the difference value by starting the counting operation of the counter during the second period from the fourth count value obtained by inverting the sign of the first count value. The photoelectric conversion device according to Configuration 1, characterized in that. (Configuration 5) The counting unit further includes a count value holding unit that holds the fourth count value, and when the difference value is equal to or less than a predetermined threshold value, the count control unit reads the fourth count value held by the count value holding unit into the counter, and controls the counter to start the counting operation during the third period of the exposure period from the fourth count value. The determination unit further performs the determination process based on a difference value between the first count value and a third count value in the third period. The photoelectric conversion device according to Configuration 4, characterized in that. (Configuration 6) When the difference value exceeds the predetermined threshold value, the count control unit initializes the count value of the counter to 0 and shifts to the count operation in the next exposure period. The photoelectric conversion device according to Configuration 3 or 5, characterized in that. (Configuration 7) After the determination process, the count control unit initializes the count value of the counter to 0 and shifts to the count operation in the next exposure period. The photoelectric conversion device according to any one of Configurations 1, 2, and 4, characterized in that. (Configuration 8) The count control unit When the difference value exceeds a predetermined threshold value, the count value of the counter is initialized to 0 and the process shifts to the count operation in the next exposure period. When the difference value is less than or equal to the predetermined threshold value, the process shifts to the next period of the exposure period without initializing the difference value held by the counter. The photoelectric conversion device according to any one of Configurations 1, 2, and 4, characterized in that. (Configuration 9) The exposure period has a plurality of sub-periods whose lengths sequentially increase by a power of 2. The count control unit performs a count operation in which a sub-period until the count value of the counter exceeds a predetermined value is defined as the first period, and a count operation in which the next one sub-period is defined as the second period. The photoelectric conversion device according to any one of Configurations 1, 2, and 4, characterized in that. (Configuration 10) The predetermined value is determined based on the maximum count value of the counter. The photoelectric conversion device according to Configuration 9, characterized in that. (Configuration 11) The photoelectric conversion unit stops outputting the pulse signal in sub-periods of the exposure period other than the first period and the second period. The photoelectric conversion device according to Configuration 9 or 10, characterized by the above. (Configuration 12) Having a plurality of pixel portions, The first period and the second period are independently set for each of the plurality of pixel portions The photoelectric conversion device according to any one of Configurations 9 to 11, characterized by the above. (Configuration 13) The determination unit has a comparator that compares the difference value with a predetermined threshold value, and determines that the light quantity has changed when the difference value exceeds the predetermined threshold value The photoelectric conversion device according to any one of Configurations 1 to 12, characterized by the above. (Configuration 14) The determination unit outputs a first value when the difference value exceeds a predetermined threshold value and the second count value is greater than the first count value, outputs a second value when the difference value exceeds a predetermined threshold value and the first count value is greater than the second count value, and outputs a third value when the difference value is equal to or less than the predetermined threshold value The photoelectric conversion device according to Configuration 13, characterized by the above. (Configuration 15) The counting unit outputs an addition value obtained by adding a predetermined threshold value having a sign opposite to the sign of the difference value to the difference value, The determination unit determines that the light quantity has changed when the second count value and the addition value are positive values or when the second count value and the addition value are negative values The photoelectric conversion device according to any one of Configurations 1 to 12, characterized by the above. (Configuration 16) The determination unit outputs a first value when the second count value and the addition value are positive values, outputs a second value when the second count value and the addition value are negative values, and outputs a third value when the signs of the second count value and the addition value are different The photoelectric conversion device according to Configuration 15, characterized by the above. (Configuration 17) The counting unit outputs the first count value or the second count value as a frame count value The photoelectric conversion device according to any one of Configurations 1 to 16, characterized in that... (Configuration 18) The length of the first period is the same as the length of the second period The photoelectric conversion device according to any one of Configurations 1 to 17, characterized in that... (Configuration 19) The photoelectric conversion unit has an avalanche photodiode The photoelectric conversion device according to any one of Configurations 1 to 18, characterized in that... (Configuration 20) A photoelectric conversion device according to any one of Configurations 1 to 19, and A signal processing device that processes a signal output from the photoelectric conversion device A photoelectric conversion system characterized by comprising... (Configuration 21) A moving body, comprising A photoelectric conversion device according to any one of Configurations 1 to 19, and Distance information acquisition means for acquiring distance information to an object from a parallax image based on a signal from the photoelectric conversion device, and Control means for controlling the moving body based on the distance information A moving body characterized by comprising... (Configuration 22) A photoelectric conversion device according to any one of Configurations 1 to 19, and An optical device corresponding to the photoelectric conversion device, A control device for controlling the photoelectric conversion device, A processing device for processing a signal output from the photoelectric conversion device, A mechanical device controlled based on information obtained by the photoelectric conversion device, A display device for displaying information obtained by the photoelectric conversion device, and A storage device for storing information obtained by the photoelectric conversion device, and at least any one of... An apparatus characterized by comprising...

Explanation of Reference Numerals

[0178] 20... Photoelectric conversion unit 30…Counter section 32…UD counter 34…Memory 36…Counter 38…Sign inversion section 40…Count control section 50…Judgment section 52…Comparator 54…Sign judgment section 100…Photoelectric conversion device

Claims

1. A photoelectric conversion unit that outputs a pulse signal in response to the incidence of light, A counting unit having a counter that counts the pulse signal output from the photoelectric conversion unit during a predetermined exposure period, A count control unit that controls the counting unit, A pixel unit having a determination unit that performs a determination process for determining the presence or absence of a change in the amount of light incident on the photoelectric conversion unit based on the count value of the counter, The exposure period has a plurality of periods including at least a first period and a second period, The count control unit controls the counting unit so as to output a difference value between a first count value in the first period and a second count value in the second period, The determination unit performs the determination process based on the difference value A photoelectric conversion device characterized by the above.

2. The counter is capable of performing up counting and down counting, The count control unit, In the first period, controls the counter to either up count or down count, In the second period, controls the counter to the other of the up count and the down count, and starts the counting operation from the first count value to generate the difference value The photoelectric conversion device according to claim 1, characterized by the above.

3. The counting unit further has a count value holding unit that holds the first count value, When the difference value is equal to or less than a predetermined threshold value, the count control unit reads the first count value held by the count value holding unit into the counter, and controls the counter to start the counting operation in the third period of the exposure period from the first count value, The determination unit further performs the determination process based on the difference value between the first count value and the third count value in the third period The photoelectric conversion device according to claim 2, characterized by the above.

4. Further having a sign inversion unit that inverts the sign of the count value output from the counting unit, The count control unit generates the difference value by starting the counting operation of the second period by the counter from a fourth count value obtained by inverting the sign of the first count value, The photoelectric conversion device according to claim 1, characterized by the above.

5. The counting unit further has a count value holding unit that holds the fourth count value, When the difference value is equal to or less than a predetermined threshold value, the count control unit reads the fourth count value held by the count value holding unit into the counter, and controls the counter to start the count operation in the third period of the exposure period from the fourth count value. The determination unit further performs the determination process based on the difference value between the first count value and the third count value in the third period. The photoelectric conversion device according to claim 4, characterized in that.

6. When the difference value exceeds the predetermined threshold value, the count control unit initializes the count value of the counter to 0 and shifts to the count operation in the next exposure period. The photoelectric conversion device according to claim 3 or 5, characterized in that.

7. After the determination process, the count control unit initializes the count value of the counter to 0 and shifts to the count operation in the next exposure period. The photoelectric conversion device according to any one of claims 1, 2, and 4, characterized in that.

8. The count control unit When the difference value exceeds the predetermined threshold value, the count value of the counter is initialized to 0 and the process shifts to the count operation in the next exposure period. When the difference value is equal to or less than the predetermined threshold value, the process shifts to the next period of the exposure period without initializing the difference value held by the counter. The photoelectric conversion device according to any one of claims 1, 2, and 4, characterized in that.

9. The exposure period has a plurality of sub-periods whose lengths sequentially increase by a power of 2. The count control unit performs a count operation in which a sub-period until the count value of the counter exceeds a predetermined value is set as the first period, and a count operation in which the next one sub-period is set as the second period. The photoelectric conversion device according to any one of claims 1, 2, and 4, characterized in that.

10. The predetermined value is determined based on the maximum count value of the counter. The photoelectric conversion device according to claim 9, characterized in that.

11. The photoelectric conversion unit stops outputting the pulse signal in sub-periods other than the first period and the second period during the exposure period. The photoelectric conversion device according to claim 9, characterized in that.

12. It has a plurality of pixel units. The first period and the second period are independently set for each of the plurality of pixel units. The photoelectric conversion device according to claim 9, characterized in that.

13. The determination unit includes a comparator that compares the difference value with a predetermined threshold value, and determines that the light quantity has changed when the difference value exceeds the predetermined threshold value. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that.

14. When the difference value exceeds a predetermined threshold value and the second count value is greater than the first count value, the determination unit outputs a first value. When the difference value exceeds a predetermined threshold value and the first count value is greater than the second count value, the determination unit outputs a second value. When the difference value is less than or equal to the predetermined threshold value, the determination unit outputs a third value. The photoelectric conversion device according to claim 13, characterized in that.

15. The counting unit outputs an addition value obtained by adding a predetermined threshold value having a sign opposite to the sign of the difference value to the difference value. When the second count value and the addition value are positive values or when the second count value and the addition value are negative values, the determination unit determines that the light quantity has changed. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that.

16. When the second count value and the addition value are positive values, the determination unit outputs a first value. When the second count value and the addition value are negative values, the determination unit outputs a second value. When the signs of the second count value and the addition value are different, the determination unit outputs a third value. The photoelectric conversion device according to claim 15, characterized in that.

17. The counting unit outputs the first count value or the second count value as a frame count value. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that.

18. The length of the first period is the same as the length of the second period. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that.

19. The photoelectric conversion unit includes an avalanche photodiode. The photoelectric conversion device according to any one of claims 1 to 5, characterized in that.

20. The photoelectric conversion device according to any one of claims 1 to 5, and A signal processing device that processes a signal output from the photoelectric conversion device A photoelectric conversion system characterized by comprising.

21. A moving body, The photoelectric conversion device according to any one of claims 1 to 5, and Distance information acquisition means for acquiring distance information to an object from a parallax image based on a signal from the photoelectric conversion device, Control means for controlling the moving body based on the distance information A moving body characterized by having

22. The photoelectric conversion device according to any one of Claims 1 to 5, An optical device corresponding to the photoelectric conversion device, A control device for controlling the photoelectric conversion device, A processing device for processing a signal output from the photoelectric conversion device, A mechanical device controlled based on information obtained by the photoelectric conversion device, A display device for displaying information obtained by the photoelectric conversion device, and At least any one of a storage device for storing information obtained by the photoelectric conversion device An apparatus characterized by comprising

Images